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Jay Earley . 

An efficient context-free parsing algorithm 


ABSTRACT 

This paper describes a parsing algorithm for context-free grammars, 
which is of interest because of its efficiency. The algorithm runs in 
time proportional to n^ (where n is the length of the input string) on 
all context-free grammars. It runs in time proportional to n on 
unambiguous grammars, and we actually show that it is n on a considerably 
larger class of grammars than this, but not on all grammars. These two 
results are not new, but they have been attained previously by two 
different algorithms, both of which require the grammar to be put into a 
special form before they are applicable. 

The algorithm runs in linear time on a class of grammars which 
includes LR(K) grammars and finite unions of them (and the LR(K) grammars 
include those of essentially all published algorithms which run in time n), 
and a large number of other grammars. These time n grammars in a practical 
sense include almost all unambiguous grammars, many ambiguous ones, and 
probably all programming language grammars. 

We present a method for compiling a recognizer from a time n grammar 
which runs much faster than our original algorithm would have, working 
directly with the grammar as it is recognized. We show some undecidability 

results about the class of grammars that are compilable by this method. 

2 

The space bound for the algorithm is n , and by using a garbage 
collector which we describe, this can be cut down to n i n a large number 
of cases. 
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The algorithm can easily be converted into a parser, which has 
the same bounds as the recognizer except that the space bound goes up 
to n^ in order to store all the parses of very ambiguous grammars. 

The time results we have obtained are only valid for a random 
access model of a computer. The n^ result is the only one which 
carries over to a Turing-machine-l ike' model . 

Our algorithm was clearly superior to the top-down and bottom-up 
algorithms in a practical comparison with data from the Griffiths and 
Petrick article. 


ill 
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|. INTRODUCTION 


Context-free grammars have been used extensively for describing 
the syntax of programming languages and natural languages. They are 
often called BNF grammars when used for programming languages CNa 63]. 

The analysis of the syntax (parsing) of programs or sentences is a 
crucial part of the implementation of compilers and interpreters for 
programming languages and of programs which "understand" or translate 
natural languages. Therefore the development of automatic parsing 
algorithms for context-free grammars has importance in these areas of 
computer science. 

We need some way of evaluating the numerous parsing schemes which 
have been developed. In order that this evaluation be unbiased by a 
particular computer, a particular method of implementation, or a 
particular set of test grammars, we will formalize our evaluation 
criteria and use mathematical rather than experimental methods for 
comparison. We are interested in how fast the algorithm runs, how 
much space it uses, and how widely applicable it is. To evaluate time 
and space, we use a formal model of an idealized computer and count 
the number of steps an algorithm takes and the number of registers it 
uses (see Section XV). To evaluate the applicability, we use a formal 
model of a grammar and consider what subclasses of grammars can be 

handled correctly by the algorithm. 

These formal techniques are some of those used by researchers in 
complexity theory— the study of the intrinsic difficulty of computing 
various functions. In fact the results in this paper can be thought of 


as complexity results. 
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In particular, we are more concerned with time than space. This 
•5 because the space requirements for context-free language processing 
generally fall within the capabilities of most large computers, but 
the time requirements often are not reasonable for the application. We 
consider the length of the string being parsed as the most important 
parameter in evaluating the time. It is more critical than the grammar 
size because the way in which the time depends on the grammar size 
seems to be pretty much the same over different algorithms and different 
classes of grammars, but the dependence of the time on the string length 
varies greatly. 

We are concerned with upper bounds on the time required for various 
algorithms on various subclasses of grammars. Specifically, if we speak 
of an n 2 algorithm for a subclass A of grammars, we mean that there is 
some number C (which may depend on the size of the grammar, but not on 
the length of the string), such that Cn 2 is an upper bound on the number 
of steps required in our model to parse any string of length n with 
respect to any grammar which is in class A. 

Those who are not interested in the formal properties of the 
algorithm need read only Sections I through IV, the definition of the 
algorithm and its implementation (pages 16-18 and 26-27), and Sections 
X, XI , XIII, XIV, XVI , and XVII. 
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II. TERMINOLOGY 

A I anguage is a set of strings over a finite set of symbols. We 
call these termi na I symbols and represent them by lower case letters: 
a, b, c. Since most interesting languages are infinite sets, we use a 
context-free grammar as a formal device for specifying which strings 
are in the set. It uses another set of symbols, the non-term i na I s , 
which we can think of as syntactic classes. We will use capitals for 
non-terminals: A, B, C. There is a finite set of productions or 
rewriting rules of the form A -+■ a. The non-terminal which stands for 
"sentence" is cal led the root R of the grammar. The productions with 
a particular non-terminal D on their left sides are called the 
a I ternat i ves of D. We wi I I hereafter use grammar to mean context-free 
grammar. 

Strings of either terminals or non-terminals will be represented 

k 

by Greek letters: a, 6, y. The empty string iis A. a represents 

k times 

a. ..a. a* = A U {a | i >_ I }. f ot | is the number of symbols in a. 

We will work with this example grammar of simple arithmetic 
expressions, grammar AE: 

E -*■ T 
E -* E+T 
T -*• P 
T -+ T*P 
P -+ a 

The terminal symbols are {a, +, *}, the non-terminals are {E, T, P}, 


and the root is E. 
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Most of the rest of the definitions are understood to be with 
respect to a particular grammar G. We write a => 8 if 3 Y> 5, n, 

A such that a = yA 6 and 8 = yn6 and A ^ n is a production. We write 
a 3 (8 is der i ved from a) if 3 strings a^, a | . ,a m (m >_ 0 ) such that 

a = a Q a , =» ••• 59 a m = 3 

The sequence a Q ,...,a m is called a derivation (of 8 from a). 

A sentential form is a string a such that R => a. A sentence is a 
sentential form consisting entirely of terminal symbols. The I anguage 
Hpf ined by a grammar L(G) is the set of its sentences. We may 
represent any sentential form in at least one way as a derivation tree 
(or parse tree ) reflecting the steps made in deriving it (though not 
the order of the steps). For example, in grammar AE, either derivation 

E =» E+T => T+T =* T+P =■ T*P+P 
or 

E => E+T =* E+P =» T+P => T*P+P 


is represented by 



E + 

I 

T 




T 

I 

P 


The degree of ambiguity of a sentence is the number of its distinct 
derivation trees. A sentence is unambiguous if it has degree I of 
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ambiguity- A grammar is unambiguous if each of its sentences is 
unamb i guous . 

The handle of a derivation tree is the set of terminal nodes which 
emanate from the left-most, lowest non-terminal node (the lower left 
» T *P» in the above tree). That is, if we think of parsing as pruning 
branches from a derivation tree, it is the first prunable branch that 

we come to, scanning from left to right. 

A .recog ni zer is an algorithm which takes as input a string and 
either accepts or rejects it depending on whether or not the string 
is a sentence of the grammar. A parser is a recognizer which als^ 
outputs the set of all legal derivation trees for the string. 

We use // as an end-of -proof symbol . 
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III. PREVIOUS WORK 

The previous work in developing efficient parsing algorithms can 
be split into two independent developments. 

Time n Algorithms . These algorithms are concerned mainly with 
parsing in compilers; they all work in time proportional to n on 
various subclasses of grammars. Perhaps the jest known of these is 
Floyd's operator precedence algorithm CFI 63]. It works by comparing 
the operators (terminals) immediately to the right and left of a point 
in a sentential form to determine whether or not it is the boundary of 
a handle. Wirth and Weber's precedence algorithm CWW 66] extends this 
to non-terminals as well as terminals and hints at extending it to 
strings as well. 

Bounded context analysis [FI 64a] allows the algorithm to look at 
a bounded number of symbols to the right and left of a possible handle 
to determine whether or not it is a handle and what it is to be parsed 
as. Knuth's LR( k) algorithm [Kn 65] extends this in that it is able to 
consider the entire left part of the string (it scans from left to 
right) plus a bounded context on the right in order to determine the 
hand I e. 

There are at least a dozen more algorithms which fall into this 
time n class. Many of these are referenced in Knuth's article. In 
attempting to compare these algorithms using our criteria, we discover 
that they all use time and space proportional to n, so the class of 
grammars which they can handle is the only distinguishing factor. 
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(Th is is not to say that it is the only distinguishing factor from a 
practical po i nt of v i ew . ) By this standard, Knuth’s algorithm is the 
best, because the LR( k) grammars include as a subset the grammars which 
the other three algorithms can handle and those of almost all other 
known time n algorithms. Recently there have been a couple of time n 
algorithms developed which seem to handle some non-LR(k) grammars 
[Ma 68] [Wi 68]. 

General Algorithms . These algorithms are general in that they can 
handle any context-free grammar. The oldest of these simply attempt to 
construct a possible parse tree from the input string in a straight- 
forward way. If the algorithm starts this construction from the root 
and works toward the input string, it is called top down [FI 64b]. If 
i+ starts from the input string and works toward the root it is called 
bottom up [lr 61]. The basic idea of either of these algorithms is to 
make arbitrary choices about what the tree might look like and to back 
up and try again if the choices later turn out to be incorrect. Although 
some selectivity about these choices can be put into these algorithms, 
the fact remains that because of the backtracking involved, grammars can 
be constructed which cause even the selective versions to run in time 
proportional to C n for any C [FI 64b]. These backtracking algorithms 

are summarized in CGP 65], 

The predictive analyzer [KO 63] is a version of top down analysis 
which works on grammars in standard form (the leftmost symbol on the 
right side of any production is terminal). However, even with a path 
elimination technique [Ku 65] to cut down on the time, it has been 



shown that it can have exponential growth in some cases [Gr 66]. In 
[Ka 66] is described an algorithm which is exponential only when the 
number of parses of the sentence grows exponentially. 

An algorithm without this exponential worst case was developed 
independently by Cocke [Ha 62] and Younger [Yo 67]. It requires that 
the grammar be put into a normal form such that every production is of 
the form A -*■ a or A -* BC; this can be done effectively for any 
context-free grammar. It then provides a recognition algorithm for 
these grammars with an upper bound of n . Younger [Yo 66] and 

Kasami [Ka 67] are also able to recognize strings with respect to 

\ 2 
linear and meta-l inear grammars (see Section VIII) in time n . And 

more recently Kasami [KT 68] has developed an algorithm which 

• +• 2 

recognizes any unambiguous grammar in time n . 

One disadvantage of these two separate developments is that none 
of the time n algorithms are also applicable to larger classes of 
grammars, and none of the general algorithms seem to do particularly 
well (time n) on interesting subclasses such as those treated by the 
time n algorithms. The algorithm presented in this paper seems to 
remedy this difficulty and unify the two developments. 
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IV. INTUITIVE explanation 

The following is an informal description of the algorithm as a 
recognizer: It scans an input string X ( ...X n from left to right looking 

ahead some fixed number k of symbols. As each symbol X. is scanned, a 
set of states S. is constructed which represents the condition of the 
recognition process at that point in the scan. Each state in the set 
represents (I) a production such that we are currently scanning an 
instance of its right side, (2) a point in that production which shows 
how much of the production we have scanned, (3) a pointer back to the 
position in the input string at which we began to look for that instance 
of the production, and (4) a k-symbol string which can legally occur 
after that instance of the production. We will represent this quadruple 
as a production, with a dot in it, followed by an integer and a string. 

For example, if we are recognizing a*a with respect to grammar AE 
and we have scanned the first a, we would be in the state set S| 
consisting of the following states: 

P -*■ a. 0 
T -*■ P. 0 
T -* T.*P 0 
E -*■ T. 0 
E -*• E.+T 0 

each with various k-symbol strings. Each state represents a possible 
parse for the beginning of the string, given that we have seen only 
the a. All the states have 0 as a pointer, since all the productions 
represented must have begun at the beginning of the string. 



There will be one such state set for each position in the string. 

To aid in recognition, we place k+l right terminators H (a symbol 
which doesn’t appear elsewhere in the grammar) at the right end of 
the input string. 

To begin the algorithm we put the single state 

<f> -*■ . R H H 0 

into state set S Q , where R i s the root of the grammar and where <t> is 
a new non-term i na I . 

In general, we operate on a state set S. as follows: we scan the 
s tates in the set in order, performing one of three operations on each 
one depending on the form of the state. These operations may add more 
states to S. and may also put states in a new state set S. +] . We will 
describe these three operations by example: 

In grammar AE, with k = I , S Q starts as the single state 

<|, .E A -I 0 ( 1 ) 

The predictor operation is applicable to this state because there is a 
non-terminal E to the right of the dot. It causes us to add one new 
state to S. for each alternative of E. We put the dot at the beginning 
of the production in these new states since we haven't scanned any of 
its symbols yet. The pointer is set to i, since the state was created 
in S.. The k-symbol look-ahead string in this case is -I, since it is 
after E in the original state. Thus the predictor adds to S. all 
productions which we might begin to look for at X. + | . 



In our example, we add to Sq 


E -*■ 

.E+T 

-4 

0 

(2) 

E *■ 

.T 

-4 

0 

(3) 


We must now scan these two states. The predictor is also applicable to 
them. Operating on (2), it produces. 


E -*■ 

.E+T + 

0 

(4) 

E -> 

.T + 

0 

(5) 


The difference is only in the look-ahead symbol. Operating on (3), 
i t produces 

T ■+ .T*P H 0 
T -*■ .P -10 

Now, the predictor, operating on (4) produces (4) and (5) again, but 
they are already in S Q , so we do nothing. From (5) it produces 

T -* . T*P t 0 

T -*■ .P +0 

The rest of Sq is 

T -► .T*P * 0 

T -»■ .P * 0 

P ■> .a -4 0 



p •> 


I 2 


.a + 0 

P -* .a * 0 

The predictor is not applicable to the last three states. Instead the 
scanner is, because they have a terminal to the right of the dot. The 
scanner compares that symbol with X. + | , and if they match, it adds the 
state to S- + |> with the dot moved over one r ~ the state to indicate 
. +hat we have scanned that terminal symbol. 

If X! = a, then S ( is 

P -+ a . A 0 

P -* a. + 6 (6) 

P -* a. * 0 

these states being added by the scanner. 

If we finish processing S. and S. +| remains empty, then an error 
has occurred in the input string. Otherwise, we then start to process 

S i + I • 

The third operation, the comp I eter , is applicable to these states 
in S, because the dot is at the end of the production. It compares the 
look-ahead string with X j+| ...X. +k . If they match, it goes back to the 
state set indicated by the pointer, in this case S Q , and adds all 
states from S Q which have P to the right of the dot. It moves the dot 
over P in these states. Intuitively, S Q is the state set we were in 
when we went looking for that P. We have now found it, so we go back 



3 


to all the states in Sq which caused us to look for a P, and we move 
the dot over the P to show that we have successfully scanned it. 

If = + , then the completer is applicable to (6), and we add 

to S | 

T -*■ P. H 0 
T -* P. +0 
T -* P. * 0 

Applying the completer to the second of these produces 

E -*• T. A 0 
E -*■ T. +0 
T -v T. *P A 0 
T -> T.*P tO 
T -*■ T. *P * 0 

and finally, from the second of these, we get 

4> -*> E. a H 0 
E -* E.+T ^ 0 


E •> E.+T + 0 



The scanner then adds to 

E ->• E+.T H 0 
E ■> E+.T +0 

If the algorithm ever ends up with S. +| consisting of the single 

state 

$ -y E H • “i 0 

then we have correctly scanned an E and the -I , so we are finished with 

the string, and it is a sentence of the grammar. 

A complete run of the algorithm on. grammar AE is on page 15. In this 
example, we have written as one all the states in a state set which differ 
only in their look-ahead string. (Thus "He*" as a look-ahead string 
stands for three states, kith V\ "t", and "»» as their respective 
look-ahead strings.) 

The technique of using state sets and the look-ahead are derived 
from Knuth's work on LR(k) grammars [Kn 65]. In fact our algorithm 
approximately reduces to Knuth's algorithm on LR(k) grammars. 



15 


Grammar AE 

root: E -*■ T|E+T 

T -► P|T*P 
P -> a 

input string = a+a*a 
k = I 


s o 

<j> -*■ . E -1 

H 

0 

S 3 

P -*• a . 

4+* 

2 


E ■+• .E+T 

4 + 

0 


T -*• P. 

4 +* 

2 


E + .T 

4 + 

0 


E -► E+T. 

A + 

0 


T + . T*P 

4+* 

0 


T ->■ T.*P 

4 +* 

2 


T -»■ .P 

4+* 

0 

S 4 

T T*.P 

4+* 

2 


~0 

Q) 

4+* 

0 


P -*■ .a 

4+* 

4 

s l 

P -*■ a. 

4+* 

0 

S c 

P ■+ a • 

4 +* 

4 


T -► P. 

4+* 

0 

0 

T -+ T*P. 

4 +* 

2 


E - T. 

4 + 

0 


E -*■ E+T. 

4 + 

0 


T -+ T.*P 

4+* 

0 


T -* T.*P 

4 +* 

2 


(f> ■+ E • 4 

4 

0 


$ + E . 4 

4 

0 


E + E.+T 

4 + 

0 


E - E.+T 

4+* 

0 

S 2 

E -»■ E+.T 

4 + 

0 

S 6 

4> -*■ E -4 . 

4 

0 


T -+ . T*P 

4+* 

2 






T + .P 

4+* 

2 






(0 

i 

CL 

4 +* 

2 







V. THE RECOGNIZER 


The following is a precise description of the recognition algorithm 

Notation: Number the productions of grammar G arbitrarily l,...,d-l, 

where each production is of the form 

D -v C ,...C - (I < p < d-l ) 
p pl PP ” 

where p is the number of symbols on the right hand side of the pth 
production. Add a Oth production 

d 0 - r h 

where R is the root of GH is a new terminal symbol, and k is a 
non-negative integer called the look-ahead parameter. 

Definition . A state is a quadruple <p, j, f, <*> where p, j, and f 
are integers (0 < p < d-l > (0 < J < ?) <0 < f I n+l ) and a is string 
consisting of k terminal symbols. A state set is an ordered set of 
states. A final state is one in which j = p. We add_ a state to a 
state set by putting it last in the ordered set unless it is alread y 

a member . 

Definition . H r (y> = {a|pt is terminal, |a| = k, and 3 6 such that 
* 

y => a0} . 

H (y) is the set of a I I k-symbol terminal strings which begin 
k 

some string derived from y . This is used in forming the look-ahead 
string for the states. 
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The Recognizer . This is a function of 3 arguments REC(G,X ( . . .X n , k) 
computed as fol lows: 

Let X n+ . = -I ( I 1 i < k+l). 

Let S. be empty (0 <_ i <_ n+l ). 

Add <0,0,0 ,«/ > to S Q . 

Set i 0 and go to A. 

A: Process the states of S. in order, performing one of the 

following three operations on each state s = <p, j> f* a> • 

(1) Predictor: If s is non-final and C ^. + |j is non-terminal, 

then for each q such that C ( j + j > = D q > and for each 

M H, (C / a) ad<j <C l> 0 > '» S> +0 V 

k p(j+2) pp 1 

(2) Completer: If s is final and a = X. +| ...X. +k , then for 

each <q, l, g, B> e S f (after all states have been added to S f ) 
such that C = D^, add <q, £+1 , g, 6> to S.. 

(3) Scanner: If s is non-final and C p(j . + |) is terminal, then 

if C P { j+i > = x i+i- add <p> J +l ’ f - a> + ° s i+i- 
If S. + | is empty, reject X ( ...X n . 

If i = n and S. + | = {<0, 2, 0, -j >}, accept X|...X n . 

Otherwise set i +■ i + l and Jo to A. 

This is not really a complete description of the algorithm until 
we describe in detail how all these operations are implemented in our 



18 


However, we will defer that description until it is required to 

mode I • 

obtain the time bounds. 

We will now prove that this algorithm is indeed a recognizer. 

Firs + we introduce the idea of i -state, which is a way of stating, 
in an algorithm-i ndependent way, the properties of a state constructed 
by the algorithm. We will use this concept in later proofs. Theorem I 
establishes the relationship between i-states and states constructed by 
+he algorithm. Theorems 2, 3, and 4 then complete the recoc :er proof. 

Qef ; nition. The extensions of a string a are the strings a0 for all 6 

Definition . The i-states of a derivation of a string X,...X n (I 1 i in) 
( let X + | = H ) are the triples (p, j , f ) , (0 <_p < d-l), < j < p), 

( I < f < i ) such that 3 i ( i < i < n+l) such that 


(a ) 

if 

P 


0, R- X r ..X f D p X t+| ...X, 





* 

(b) 

i f 

j 

> 

0 , c , . . . c . =* x , , . . • x . 

'pi PJ f+l 1 





* 

(c) 

if 

j 

< 

P’ c p ( j+l ) — c pp * x i+l ■ 


in that derivation. 

Def i nition . The i-states of a string X ( . . . X m (m > i) are the i-states 

of the derivations of the extensions of X|...X m> 

The i-states of a string X ( ...X m are roughly those states constructed 
by the algorithm'on X|...X. which are consistent with X. + |.«*X m being 


next in the input string. 
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No+e; Un |ess otherwise specified, the algorithm is understood to be 
REC(G, X r ..X n ,k) in our proofs. 


Theorem I ■ If <P , j. f > a> e S i and 

x i+ l*-- x i+k e H k (C p( j+l ) ' * ’ C PP a) 
then (p, j, t) is an i-state of X ( ...X j+k . 

Proof: By induction on the number of states added to any state set 

before <p, j, f, <*> Is added to S . . 

Basis: <0, 0, 0, A> is the first state added to S Q . If X | ...X |< e 

H (R-» k+l ), then (0, 0, 0) is an i-state of X, — X R as follows: 

k 


(a) p = 0, so not applicable 


(b) j = 0, so not applicable 

(c) 3 0 such that R =» X....X 6 since X |«*- X k e H k (RH ) 


Induction Step: 

(I) <q, 0, i, e> is added to S. by the predictor from state 

<p, j» f » a> and 

X ...X. , e H.(C ,...C - 0) = H (D 0) 
i+| i+k k ql qq K M 


Then 

X i+|* ,,X i+k e H k (C p(j+l ) C p(j+2)‘ 
since D q = C p(j+1) and 0 e H k CC p( j+2) . . .C p - ct) . 



So by inductive 
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hypothesis (p, j, f) is an i-state of X r ..X. +k< 
extension X ( ...X n and an SL such that 


Therefore 3 an 


R - V-*x f D p x 1 +| ...x n 


'pi 


.c 


PJ 


f+l 


.X. 


and C p(j+I )- ,,C pp 19 X i + I 

So there is an £’ (i such that 

C P ( j+2) * * ,G pp ^ x 4 ’+r'' x «- 

I f 3 y such that 

V" C qq ix i + r*‘W 

(q, 0 , i) is an i-state of a derivation of X j • • * X j + k YX j, * + 1 • ' ,X n aS 
fol lows: 

(a) D p =» X f+| ■•• x j D q V + l ...X r so 

R => X! . . *Xj DpX^, + j . . .X r 

(b) j = 0, so not applicable 

(c) C q I ‘ ” ‘ C qq ^ X i + I ’ ’ * X i +k^ 
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If 3 k' < k such that C q| . . .C q - * x i+t • • - X j+k’ » +hen (q ’ °» 1 } 
is an i-state of a derivation of X, . . • x i+k ,X jl , + | • • - x n (l+k ' = V) 
aS foi lows: 

(a) and (b) are the same as above. 


(c) C q I * * * C qq * X 1 + 1 — X l + k* 


(2) <q, j+l, g, 6> is added to S. by the completer from 
<p, p, f, a> e S. and <q, j, g, B> e S f . Then a = X. +) ...X. +k , so 
by inductive hypothesis, (p, p, f) is an 


i -state of X .... X. 


i+k* 


So 


Also, since 


C . , • • c — ^ x , , | ♦ . .x . 

pi PP f+l • 1 


V,--- X itk*V C q<jt2>--- C qq 61 


X f + |--- X i X i + l"- X i + k e H i-f+k <C pl'" C pp C q(j+2)"' C qq B> 


which equals H._ f+(< <C q(j . + |) 


...C_ r S) since D p = C q(j+|) . So 


qq 


X f+| — X f+k e H k (C q( j+l ) * " "qq 


..C - 8). So by inductive hypothesis (q, j, g) 


is an f-state of X, ...X f+|< . Therefore 3 an extension X r ..X n and an l 


R * X l"- X g D q X WI"- X n 


C . . . .C . ^ X | . . .x, 
ql qj 9 +l f 


such that 
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and 


C p ( j+l ) ’ ’ ,( "pp X f + 1 ‘ 


| f 3 Y such that 


C p( j+2) * * * C pp * X i + I " ' X i+k Y 


(q, J+l , g) is an i -state of a derivation of X ( . . .X r+k Y X £ + | • • - X n as 
fol lows: 

(a) R=* x | • * X g D q X Jl+ 1 * * * X n 

(b) C q i * ’ * c q j c P r- c P r* Vr-'VW-- X i 50 

c q |- C q( J+ l) 1 Vl-V 5in « C q<j + I) = °P 

(c) C p ( j+2) * ' * C pp * t + l’"' X i + k 

% 

If 3 k’ < k such that C p( J+2) • • *C p - =» x i+ | • • - X i+k’ ’ +hen 
(q, j+l, g) is an i-s+ate of a derivation of X | • • • x j +k t x Jl+ | • • ,x n 

(i+k' = l) as fol lows: 

(a) and (b) are the same as above. 

* 

(O C p(j+ 2 ) ...C p -=> x i + | ••• X i + k ’ 

(3) < p , j+|, f, 0> is added to S. by the scanner from 

<p, j, f, B> e S._ ( . 


X i + 1 ’ ’ ,X i + k 6 H k (C p ( j+2 ) * * * C pp 6) 
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and C p(j+I ) 
so 

Therefore, 
X | ' • ,X i+k-l 


and 

so (p, j+l , 

(a) f 

(b) ( 

(c) f 

Theorem 2 . 
sentence. 

Proof: If 

{< 0 , 2 , 0 , 


= X. , so 

i 


X X. 


i+k 


H k+I (c p(j+l )• 



V-- x i+k-i e H k (c P ( j+ 1 ) * * * c pp e) 


oy inductive hypothesis, (p, j, r; is an (i-i)-state of 
. Therefore 3 an extension X,...X n and an l such that 


R =» X | • ' ' X f D p X i,+ l • * * X n 


C ,...C X, | ■ ■ iX, . 
pi PJ f+l '“I 


C ^ x. . . .X 

p(j+l) pp I l 


f) is an i-state of a derivation of X|...X n as follows 


* * X, . . -XfDpX^+i • • - x n 


"p I * ‘ * C P j C p (j+l > ^ X f+I * ‘ ,X i-l X i SInCe C P(J>D 

: p(j+2)* ,,c pp * x i+r**V 11 


= x. 


If X ...X is accepted by the algorithm, then it is a 
I n 

X . . .X is accepted by the algorithm, then S , - 
In 

H^>}. And by theorem I, since H € (0, 2, 0) 
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. _ (n+n-state of X....X - 4 k+l . So by (b) i n the def i ni tion of 

i s an I n 

i -state, 

R H X! . . .X n H 


or 


.X , so it is a sentence. // 
n 


Theorem 3 . 
X 5 ,+ | • • ' X £+k 


If <p, j, f, «> e s ;» C p(j+I ) ^ X i + I * “V and 
€ H k <C p ( j + 2) ' ’ " C pp the " <P ’ j+l> f> £ V 


Proof : 
C p(j+I) 


By i nduction on m 
# 


X. 


i + l 


.X 


l' 


(see page 4) 


? * i 

in the definition of =* in 


Basis: I f m = 0, then l - i + l and C p(j . + | ) - x i+ |» 30 < P» J +l » f » a> 

is added to S. +| by the scanner. 


I nduction Step: i f m > 0, 3 q such that c p ( j+| ) D q C ql’ * ,C qq 

and 3 + 0 i +| !•••! + q such +ha+ + 0 = ’ ' + q = £ and 


C 


qi 


V 




Vn +I 


.X 


t- 

q 


Because of the predictor acting on <p, j, f, <*> in S., 

<q, 0, i, X A+| ...X 4+k > will be added to S. since 
y X e H (C „,...C - a). And by inductive hypothesis 

vr^Vk 6 V ptj + 2) pp 

we know that if <q, r, i, , - * * X £+k > is in S t r ’ +hen <q> r+i ’ ' 

X ...X. > will be in S, (0<^r<^q-l)si nee 

4+1 4+k t (r+|) 
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* 

C q ( r+l ) * X t p + 1 " * X+ ( r-t-l ) 

and X t, M + r ,,X t, iM +k s H k (C q(r+2)’ ,,C qq Vr’Vk’ 

(r+l ) (r+l ) M 

So by induction on r, S^._ = contains <q, q, i, X^ + | . . .X^ +k >. 

q 

And the completer, acting on this state, adds <p, j+l, f, a> to S . // 

Theorem 4. lfX,...X is a sentence, then it is accepted by the algorithm. 
— I n 

Proof : S n contains <0, 0, 0, -I k >, R =* X...X, and H k e H. H k *). So 

U I n k 

k 

by theorem 3, S n contains <0, I, 0, H >. And since H = C Q2 , the 
scanner adds <0, 2, 0,H k > to S n+| . This is the only state that S n+| 
can contain since -f appears nowhere else in the grammar. So X | . . . X^ 
is accepted. // 

Theorems 2 and 4 together show that the algorithm is a recognizer 
for all context-free grammars. Notice that our recognizer proofs 
require no restrictions at all on the grammar. That is, unlike most 
algorithms, ours handles correctly circular grammars, disconnected 
grammars, grammars which generate strings with an infinite number of 
parses, even grammars which generate the empty language. 
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VI. RECOGNITION TIME 

Now we will try to characterize precisely the speed of the algorithm. 

Firs+ we describe its implementation. The formal model of a computer 
which we are using has been chosen purposely to reflect as much as 
possible the random access properties of real machines. Therefore it 
is not restricted to the square-at-a-time movement of a Turing machine. 

We will not describe our model in detail here; that will be done 
in Section XV. Instead we wi I I describe the implementation as if it 
were done on a real machine. Our implementation description assumes a 
knowledge of basic list processing techniques. 

Implementation 

(1) For each non-term i ha I , we keep a linked list of its alternatives, 
for use in prediction. 

(2) The states in a state set are kept in a linked list so they 
can be scanned in order. 

(3) In addition, as each state set S. is constructed, we put entries 
into a vector of size i. The fth entry in this vector (0 < f 1 i ) is a 
pointer to a list of all states in S. with pointer f, i.e., states of the form 
<p, J, f, ot> e S. for some p, j, a. Thus to test if a state <p, j, f, «> 

has already been added to S., we search through the list pointed to by 
the fth entry in this vector. This takes an amount of time independent 
of n. The vector and lists can be discarded after S. is constructed. 
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(4) For the use of the completer, we also keep, for each state 
set S. and non-terminal N, a list of all states <p, j, f> a> e S. such 
that C p(j . + | j = N. 

(5) If the grammar contains null productions (A - A), we cannot 
implement the completer in a straightforward way. When performing the 
completer on a nil state (A a i ) we want to add to S. each state in 
S. with A to the right of the dot. But one of these may not have been 
added to S. yet. So we must note this and check for it when we add more 
states to Sj. This does not change the time bounds, however. 

Now we will derive an upper bound on the recognition time. First 
some definitions. 

Definition . Let m-l be the maximum p (0 <_ p <_ d-l ) . 

Let a be the maximum number of alternatives of any non-termina 

Let t be the number of terminals in G. 

Let u be the number of non-terminals in G. 

Let v = t+u be the number of terminals and non-terminals in G. 

Let C be any constant independent of the length of the string. 

Definition . A sfe£. is an algorithm which takes a fixed number of basic 
operations in our model independent of the size of the grammar or of the 
string being recognized. 

The following theorem establishes the fact that our algorithm is an 
recognizer in general. 
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3 2 

Theorem 5. There is a bound of the form Cn +0(n ) on the number of steps 
required to compute REC(G, X j . . . , k) for any k. 

proof: (a) There are at most dmit k states in any state set S._ ( , so it 

takes at most dmit k steps to scan them. The scanner adds just one state 

for each state to which it is applicable. The predictor adds at most 

a + k states for each state to which it is applicable. (The members of 

can be calculated independent of the recognition process.) To test each 

state added to see if it is already in the state set costs at most dmt . 

So scanning the states in S. plus the scanner and predictor operations 
k k k 

takes at most (at ) (dmit ) (dmt ) steps. 

(b) The completer performs more than one step at a particular state 

<p, j, f, ot> only if j = p and a = X. +| ...X. +k . For each of these states 

it adds at most dmft k <_ dmit k states, one for each possible state in S^. 

It takes at most dmt k steps to test each state added. So the completer 

k k 

performs at most (di) (dmit ) (dmt ) steps. 

Summing over all state sets, we get 


n+ 


E (admit^ k + d^mi^t k ) (dmt k ) 


1 = 1 


.2k 


n+ 1 


n+ 1 


= Cadmt^''" E i + d^mt k E i (dmt ) 
1 = 1 i = I 


= [admt 2k ISiliiJSiL + d 2 mt k (l n+ ' ! > <n+2) . (*, +3).-, (dm+ k, 


. 3 2 2k 3 ,, 

'v d m t n // 


2 


6 
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Since this bound holds for any k (including k - 0) we could do 
without the look-ahead here. It will be more useful in obtaining time 
n bounds (Section IX). The n 3 bound is no better than that obtained by 
Younger [Yo 67], but the algorithm is better in that it does not 
require the grammar to be put into any special form (Younger's requires 
normal form). 
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VII. BOUNDED DIRECT AMBIGUITY 


We next ask the question: What classes of grammars can be done 
in time n^ by the algorithm? We will define one such class in terms 
of the amount of ambiguity of the grammar. 


Definition. The degree of direct ambiguity of a non-terminal A with 
respect to a string X ( . . . is the number of distinct pairs <p, t>, 
where p is a production such that D p = A, and t is an (p+l)-tuple 

<q,,....q- + l > such +ha+ 3| = °» %+] = n and 


* 


c . =» X 
PJ 


q 


,+ i “ 

j 


.x 

q (j+i ) 


(i < j < p ) 


Intuitively, the degree of direct ambiguity is the number of 
"top-level parses". That is, instead of considering all the different 
parses of X ( ...X n as A, we consider only the first production of each 
parse, say A -*■ BC, and the substrings of x | • • • x n which the symbols in 
this production generate, B =» Xj...X m , C =» X m+ 1 • * • x n • Each such 
distinct pair <p, (0, m, n)> increases the degree of direct ambiguity 

by I . 

We know the following concept: 

Definition. A grammar has bounded ambiguity if there exists a bound 
for the grammar on the degree of ambiguity of any of its sentences. 
The analogy-.of this definition for direct ambiguity is as 


fo I I ows : 
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VIII. TIME n 2 

Now we attack the n 2 question. The completer is the on I y thing 
which forces us to use n 2 steps for each symbol we scan, making the 
whole thing n 3 . So we ask, in what cases does this operation involve 
only n steps instead of n 2 ? If we examine the state set S. after the 
completer has been applied to it, there are at most proportional to i 
s + a t es in it. So unless some of them were added in more than one way 
(this ca-n happen; that's why we must test for the existence of a state 
before we add it to a state set) then it took at most * i (proportional 
to i) steps to do the operation. 

In the case that the grammar is unambiguous, we can show that each 
such state gets added in only one way, but using the concept of bounded 
direct ambiguity, we can get an even more powerful result. Lemma I 
shows that the number of ways that a state can be added is proportional 
to the degree of direct ambiguity. Using this, we prove theorem 6, 
which gives us the n 2 result for all BDA grammars. 

Lemma [ . If b is a bound on the degree of direct ambiguity of D q with 

respect to any string, then any state <q, J+l, f# a> which IS added + ° 
S by the completer can be added in at most ba different ways. 

Proof: Assume that the state <q, j + I > f» a> ' s added to S. in two 
different ways by the completer. Then we have 

s, = <P|» P r V x i + i ••’ X i+k > e s i 
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and 


s 2 = <p 2 , p 2 , f 2 , x i+l ••• X i+k > e S i 


<q » j, f» a> e s f and S f 2 


D = C 


q( j+l ) 


= D 


and either P| / p 2 or f, t f 2 , for otherwise s, and s 2 would be the 
same state. 

Consider the degree of direct ambiguity of with respect to 
X f+| ...X.B, where C q{ j+2) . . .C q - - B. It must be at least 2, since 
(in the definition of degree of direct ambiguity) f , -f and f 2 -f are 
the (j+l ) t h members of two (q+l)-tuples which partition X f+) ...X.B. 

And each other distinct f. indicates the existence of another distinct 
n-tuple. So there can be at most b such integers, since b is a bound 
on the direct ambiguity. 

If f = f 2 , there can be at most a distinct productions p. since 
each is an alternative of Cq(j+|)* Therefore <q, j + l, f» a can 


added in at most ba ways by the completer. // 

2 

Using this lemma, we can obtain the n result: 


Theorem 6 . There is an upper bound at the form Cn 2 +0(n) on the number 
of steps in computing REC(G, X. — X p , k) for any k and any BDA grammar. 
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Proof: Let G have degree b of direct ambiguity. 

(a) of theorem 5 still holds here. 

(b) At most dmit k states can be added by the completer. By 
lemma I, each can be added in at most ba different ways. And it 
costs at most dmt k to test each one. This gives us (badmit k ) (dmt k ) 
steps required for the completer. 

Summing for i = l,...n+l produces 


n+l 9k k k N 

£ (admit +badmit )(dmt ) 

i = I 


< <b+l)<admt 2k ><dmt k > "s' 1 =■ (b+ I > <adm+ 2k > <dmt k > <n+l > < n + 2>/2 

i = l 


n, . 2 2,3k 2 

"v bad m t n 


We will now show that our algorithm also achieves Younger's n results 
for linear and meta-linear grammars QYo 66]. 

Definition . A production A + a is linear if a contains at most one 
non-terminal symbol. 


Definition. A grammar is I i nea 


r if all its productions are linear. 


Definition. A grammar is 


meta-linear if no production A ^ a has the 


root R in a, and if all productions A - a where A t R are linear. 
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Theorem 7. A linear grammar has bounded direct ambiguity. 

Proof . A non-terminal A has degree of direct ambiguity of at most a 
(the maximum number of alternatives of any non-terminal) with respect 
to any string. For each production A -*■ C p| ...C p - where C pj . is the 
non-terminal (I < j < p> and each string the only (p + l)-tuple 

that partitions the string is <0, l,...j-l, j+n-p, . . . n- I , n> . // 

Combined with Theorem 6, this gives us the n 2 result for linear 

grammars . 

Theorem 8 . There is a bound of the form Cn 2 + 0(n) on the number of 
steps in computing RECCG, X r ..X n , k) for any k and any meta-l inear 

grammar G. 

Proof: (a) of theorem 5 holds here. 

(b) of theorem 6 holds also for all the linear productions 
in G, by theorem 7. However, since R cannot occur on the right side 

of any production, all states <p, j, f, <*> such +ha+ D p = R W ' M have 
been created originally (l.e., when j = 0) in S Q . Therefore f = 0 for 
all states with non-linear productions, and therefore there can be at 
most dmt k of them in S._ r Each of these states can be added a full ai 
different ways by the completer (see proof of lemma I). So we must add 
a third term, admit\ to the calculations of theorem 6. This however 
also results in an n 2 term, so we still have the n 2 result. // 

So we have achieved the n 2 results for both unambiguous and 
meta-l inear grammars with one algorithm. In addition, our algorithm does 
not require a normal form grammar while both Kasami’s and Cocke's do. 
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To illustrate these results we will run some example grammars. 
Grammar BK (page 39) is obtained by simplifying grammar K. It has 
the same ambiguity properties as K (but the language doesn't); that 
is, it has BDA, but unbounded ambiguity. Notice that state 

K -»■ KJ„ -|x 0 

in S. gets added twice by the completer. (Th.s is signified by the 
superscript on the 0.) Th i s i s because the grammar has degree 2 of 
direct ambiguity. Actually this grammar not only doesn't require 
more than time n 2 , it requires only time n, since all the state sets 

after S| are more or less the same. 

Grammar BP (page 40) is obtained in a similar way from grammar P. 
It has unbounded direct ambiguity. This is illustrated by the 
increasing superscript on state 

P -* MN. 0 

The superscript will continue to be i+l in S. . However, this is not 
enough to make this grammar require n 3 , since it is meta-l inear. This 
fact means that the state which gets added ^ i times, will be unique 
(or there will be a bounded number of them) as it is here, so the time 

i s sti I I n 2 . 

Grammar UBDA (page 42 ) is not so nice. It has unbounded direct 
ambiguity and no saving grace. One can tell by the looks of the 
exponents on states 

t 

4 
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A ->• AA. 2 

,2 

A ^ AA. I 

A -> AA. 0''’ 

in S 4 that there are ^ i states, each of which is added ^ i times. So 
this really takes time n 3 . 
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Grammar BK 




root: K -*■ a | KJ 






J + F| 1 






F - x 






1 -*• X 






sentences: x n (n 

i 0) 





REC(BK,x n , 

11} 





Sq 4> -► -KH 

H 

0 S. (2 <_ i 

< n) F -> x. 

-lx 

i - 

K . 

H x 

0 

I F + x. 

H x 

i - 

K -*■ .KJ 

-1 X 

0 

J -*■ F • 

A x 

i - 

(j) -* K. -* 1 

-I 

0 

J -> 1 • 

Hx 

i - 

K K. J 

-lx 

0 

K -* KJ . 

Hx 

o 2 

J . F 

Hx 

0 

K + K.J 

H x 

0 

J ■> • 1 

-lx 

0 

4> KH 


0 

F -> .x 

-lx 

0 

J + .F 

-lx 

i 

1 -* .X 

-lx 

0 

J ■> • 1 

9x 

i 

S. F x. 

Hx 

0 

F ■> .x 

H x 

i 

l 

1 X. 

-lx 

0 

1 .X 

Hx 

i 

J -> F. 

-lx 

0 S 

n+ 1 

<J> K H • 

H 

0 

J I • 

Hx 

0 




K -*• KJ • 

Hx 

o 2 




K ■> K.J 

Hx 

0 




+ K . H 

H 

0 




J - .F 

Hx 

1 




J • 1 

Hx 

1 




F -*■ .x 

H x 

1 




1 ■> .X 

Hx 

1 
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Grammar BP 


root: P -*■ MN 

M -> A | Ma 
N ->■ A | Na 

sentences: a n (n ^ 0) 

REC ( BP , aa, 0) : 

S d> -*• .P-l 0 

0 9 

P -> .MN 0 

M . 0 

M -*■ .Ma 0 

P -> M.N 0 

M -+ M.a 0 

N . 0 

N ■> .Na 0 

P ->■ MN. 0 

N -*■ N.a 0 

4> -*■ P.^ 0 


S | M -*■ Ma . 0 

N -*■ Na. 0 

P ^ M.N 0 

M M. a 0 

P MN. 0 2 

N -*■ N.a 0 

N ->• . I 

N ■> .Na I 

<j> -v P.H 0 

N -* N.a I 
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M Ma . 0 

2 

N -*■ Na. 0 

N -*■ Na . I 

P M.N 0 

M ■> M.a 0 

P -*■ MN. 0 3 

N -> N.a 0 

N -*■ N.a I 

N . 2 

N .Na 2 

<p -*■ P.H 0 

N -*■ N.a 2 

s 3 <p -*■ P-I. 0 
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Grammar UBDA 

root: A -*■ x | AA 

sentences: x n (n > I ) 


RECOJBDA, x 4 , I): 


(J) -* . AH 

H 

0 

X 

+ 

< 

CO 

Hx 

2 

A .x 

H x 

0 

A AA « 

Hx 

1 

A - .AA 

Hx 

0 

A ^ AA. 

Hx 

o 2 




A -*■ A. A 

Hx 

2 

A -* x. 

Hx 

0 

A ->• A. A 

Hx 

1 

4> A .H 

-1 

0 

<P A.H 

H 

0 

A A . A 

Hx 

0 

A A. A 

Hx 

0 

A -*■ .x 

Hx 

1 

A -► .x 

Hx 

3 

A .AA 

Hx 

1 

A ^ .AA 

Hx 

3 

A -> x. 

H x 

1 




A AA . 

Hx 

0 




A -*■ A. A 

Hx 

1 




4> A 

-♦ 

0 




A -* A. A 

Hx 

0 




A -> .x 

Hx 

2 




A -*■ .AA 

Hx 

2 
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X 

< 

Hx 

3 

A + AA. 

-»x 

2 

A -*■ AA. 

Hx 

I 2 

A -*■ AA . 

Hx 

o 3 

A ■> A. A 

Hx 

3 

A -* A. A 

Hx 

2 

A -> A. A 

H x 

1 

<j> -► A ,H 

H 

0 

A -> A. A 

Hx 

0 

A ■> .x 

H x 

4 

A . AA 

Hx 

4 

<p -*• AH. 

H 

0 
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IX. TIME n 


We would now like to characterize the class of grammars which the 
algorithm will do in time n. We notice that for some grammars the 
numbers of states in a state set can grow indefinitely with the length 
of the string being recognized. For some others there is a fixed bound 
on the size of any state set. We will call the latter grammars 
"bounded state" grammars, and we will show that they can be done in 
time n. 

We can define "bounded state" independently of the algorithm using 
the concept of i-state which we defined earlier (Section V). 


Def i n i t ion . A grammar is bounded state with look-ahead k, BS ( k) , if 
3 a bound on the number of different i -states of any string 



.X. (0 < i < n) . 
i + k — — 

Theorem 9 shows that our definition of bounded state really means 


what it says. 


Theorem 9 . If b is a bound on the number of i -states of any string 

m 

X....X.,,, then dmb t is a bound on the number of states in S.. 

I i+k' i 

m k 

Proof: Assume that there are dmb t +1 states in S.. Then 3 

p, j , f | , • • - , f 1 such that the f's are distinct and 

<P» a>,...,<p, j, f b m +| , a> e S. 

If j = 0, then f = i for all t (I <_ t <_ b m + 1 ) . So j ^ 0. In fact, 

if C ,...C . is a terminal string, then f, = i-j for all t (I < t < b m +l). 
pi pj + j - - 
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So one of C p C pj must be non-terminal. Let h be the largest integer 
h 1 J such that C p h is non-terminal. We know that <p, h, f , a > e s 

m 1 " 9 g 

(l - t 1 b +1) where g = i-(j-h). So these states were added to S bv 

g y 

the completer acting on some number r of final states 

X g+| . . .X > e S (0 <_ Z <_ g ) ( | < s < r) where D = C 

- - q ph 

The look-ahead string for these states must be X ,...x for the 

g+l g+k Tne 

completer to be applicable. 

By theorem I, (q, q, z ) is a g-state of X ...X . (I < s < r) 

so r < b since there can be at most b of them.- But since there are 
b m +l states added by the completer from these r states, at least one 
of the final states must have b m " l +| parents. That is for some s, 

S £s contains (p, h-l, f + , «) for b^' + l different t's. We now apply 
the above argument again as many times as there are non-terminals in 
C pl * ’ * C ph' reducin 9 the power of b by one each time. However, there 
can be at most m non-terminals, so we end up with at least b°+l (or 2) 
states <p, h\ f + , a > in some state set S * (0 <_ h' <_ h) (0 < g* < g ) 
where C p| ...C ph , is terminal. But then the two f + 's must both be g-h'. 

But the f's are all distinct. Contradiction. // 

From this, we easily obtain the time n result for bounded state 
grammars. 

— ? r9m l0 - There ls a t'Ound of the form Cn on the number of steps 
required to oompute RECtG, k) for any BS(k) grammar G. 
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Proof: By theorem 9, dmb m t k is a bound on the number of states in any 

k, it) k k 

state set S.. This changes (a) of theorem 5 to (at )(dmb t Mdmt ). 

In (b), we have at worst (dmb m t 1< )^(dmt k ) . Since both of these are 
constant with respect to n, we get 

n 

£ C = Cn // 
i = I 

Now, how does the bounded state grammar concept compare with 

other time n classes of grammars, especially the LR(k) grammars? It 

turns out that almost all LR(k) grammars are BS(O) grammars. In fact 

all except certain right recursive LR(k) grammars are BS(O). (A right 

# 

recursive grammar is one in which 3 a non-terminal A such that A 3 aA 
for some A.) However, even very simple right recursive grammars are 
not BS(k) for any k. 

The run on grammar RR with k = I illustrates this (page 48). 

Notice that all state sets have 4 or less states except which has 7. 
If the input string were x 11 , all states would have 4 or less except S n 
which would have 4+n. Grammar RR is, in fact, LRU), but there is no 
bound on the size of S^, so it is not BSU). One can see that no 
finite k wou I d remedy this. 

However, notice that each of the extra states in S actually 
represents a step in the derivation of x n . So since the derivation can 
have at most ^ n steps,' the total number of "extra states" is ^ n, and 
therefore the number of states is ^ n even though there is no bound on 
the number in a single state set. Because of this, the algorithm will 
recognize with respect to RR in time n if k = I. We wi I I prove that 
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Grammar RR 

A -*■ x | xA 

sentences: x n (n > I ) 


REC(RR,xxx,Q) : 

S Q ♦ .AH 0 

A + .x 0 

A -* .xA 0 

S j A -*• x. 0 

A -*■ x . A 0 

4> A . — I 0 

A -*■ .x I 

A -*■ . xA I 

S 2 A -*■ x. I 

A -*■ x. A I 
A -*• xA. 0 

A -* .x 2 

A -*■ .xA 2 

4> ■+■ A.H 0 


A -*■ x. 2 

A -*■ x.A 2 

A -* xA. I 

A ■> .x 3 

A -*■ .xA 3 

A -*• xA. 0 

-** A 0 

S 4 <t> + Ai 0 
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REC ( RR , xxx , I ) : 


S Q 4> -*■ -AH -(0 

A -*■ .x -i 0 

A -*• .xA H 0 

S A -*• x. HO 

A -*• x.A ^ 0 

A -*■ .x H I 

A -*■ .xA H I 

S 2 A ■> x. I 

A x.A -I I 

A -*■ .x 2 

A •> .xA ^ 2 

A -*■ x. “• 2 

A -*■ x.A H 2 

A -> xA. -I I 

A -*■ .x -t 3 

A -*■ .xA 3 

A ■> xA. ^ 0 

4> -*■ A. H -i 0 

S 4 <J> -*■ A H . -I 


0 
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this holds true for all LR( k) grammars. That is, even if they aren't 
bounded state, they ere time n if a look-ahead of k.us used. 

First we rephrase Knuth's definition of LR(k) []Kn 65]]. 


Def i n i tion . For an unambiguous grammar, (m, p) is a handle of A|...A n if 


(a) R =» A. . . .A D A . . . . .A (r = m-p) 
I r p m+l n 


(b) D ' -*• A .... .A (r = m-p) 
p r+ I m r 


7T 

and (c) i a =? A. . . .A . such that R 3 aA . . .A 
r ^ I m- 1 m r 


Def i n i tion . A grammar is LR( k) as fol lows: 

Let a = X| . . .X. + ^ot' 

8 = X, . . .X j+k 8' 

where X. _ Ll . . .X. . . , a', 8' are terminal, 
i+l i+k 

If (i, p) is a handle of a and (j, q) is a handle of 8, then i = j and 
p = q. 


Now we define the idea of a "useful" string. Given X ( . . . X . A ( . . . A^ 

is useful if it is the leftmost portion (generates X....X.) of a 

sentential form of X ( ...X n , and if it is the topmost such string in the 

derivation. Lemma 2 proves that this string is unique in LR(k) grammars. 

Note that it is also important that there is only one derivation of 

X .... X . f rom A .... A . 

I m 
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Def i n i t i on 


. A string A ...A is usef u 1 if 


m for some 5 


(2) A, . . .A m ® X| . . .X; in only one way 


(3) } T ® A ...A m such that R * Y x i + 1 . 


.X. ,6 for some 6 
i +k 


Lemma 


2. If G is LR(k), then for each string X ( ...X j+k , 3 at most one 


useful string A. . . .A m . 


Proof: By induction on 


Bas i s 


: | f i = 0, then m = 0 and A|...A m A< 


Induction Step: Assume i t i s true for i . Then the unique useful 

string for X,...X. +k> A,...A m , is the only string such that either 
(I) A ...A X. + | . . .X. +k+ |<5 is a sentential form which can have a 
handle at m+l for some 6, or (2) A,...A m X I+| is a useful string for 

X|...X. + k+ |* Le+ x ; + | " A m +r 

A: If (m+l, p) is not a handle of A, . . .A m+| X j+2 . . .X. +k+] 6 for 

any p, 5, then A,...A m+| is the only useful string for V** X i + k+r 

If (m+l, p) is a handle of A { . . .A m+ ] X. +2 . • • x ; +k+ | 5 for some 

p, 6, then it is the handle for all 5 since the grammar is IR(k). 

Therefore A.. ..AD (where r = m+l-p) is so far the topmost string 
I r p 

such that A,...A r D p ^ A r ..A m+| , and A, . . .A r D p X i+2 . . .X. +k+) 6 is a 
sentential form which can have a handle at r+l . Let A r+) - D p , 
le+ m = r . Repeat the above process starting at A until (m+l, p> 
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„ not , handle- Notice that there is only one derivation of the ne. 
A _ Am+| from the old A, ...A m X. + | , so by inductive hypothesis there 

is only one derivation A j . . « A m+ | ^ X | * • * i + l * 

Since the "extra states" above the bound on the size of a state 

set in LR(k) grammars are added by the completer, we will concentrate 
on this operation in our proofs. Lemmas 3 and 4 are used in several 

places later. 


Lemma 3 . If G is an LR(k) grammar, then 3 at most one state 

- . y V > e S which has not been added by the completer. 

<q, q, l, X. +| • • - A i+k | 

Proof: By theorem I, (q, q. A) ^ an I -state of X,...X J+k( so 

D -> C . . . -C - 39 X - . . -X. 

q q I qq 16+1 1 

This derivation then is part of the unique derivation A, . ..A m - X, .. .X, 
of the useful string A,...A m of X,...X |+k (because of (3) in the 
definition of useful, and lemma 2). But since this state was not added 
by the completer, it must have been added by the scanner, so C q - = V 
If there is another such final state <q , q > ^ ^ | + I i+k 

also part of the same derivation and has C q t“» - X | for 
reasons. Thus X. is parsed in two different ways as follows:^ If 
q * q », then X. is derived by different productions. If q = q, but 
4 + v then the production which X. is derived from generates 2 
different length strings. In any case, if the states are not identical, 
then they come from different derivations. // 
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Lemma 4. If G is LR(k), th- at most one state <q, q, l, x ; + 1 


» X . , , > 
i + k 


can 


be added to S. by a single completer operation, 


Proof: Suppose <q, q, l, X. +| ...X. +k > is added by the completer from 


<P 


, P, f, X i + |...X i+k > 6 s. and <q, q-l, X 


• .X, , > e S, 


i + l i + k 


Then 


by theorem 


, (q, q, l) and (p, p, f) are i-states of X^.-X 


i + k' 


So 


D C , . • • C - X 0 , | • • • X . 
q q! qq &+I 1 


and 


D C . ...X. 

p pi pp f+l i 


But then 


X f+l‘*’ X f+k e H k (C qq X i+l'’* X i+k ) 


so (q, q-l, l) is an f-state of X | ...X f+R . Therefore 

C q I ’ * * C q (q-l ) * X £+l - *' X f 

But all these derivations are part of the unique derivation 

A. . . .A =» X . . .X. 

I m I i 


of the useful string A | * * * A m • So since c qq “ D p » 

D - C ,...C ,- n D n 4 X,,....X If there is another state 
q q I q Cq— I ) p 1 

, -f .1 v x. added by the same completer operation, 

then by the same argument as above, 

°q» C q’ I* * ,C q^(q'-l ) °p ^ X S.' + r * ,X i 
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But by the same argument as in lemma 3 (using D p instead of X.), this 
is impossible. // 

Now we will show that the number of states added by a single 
completer operation is bounded (lemma 5). Then we show that the number 
of states in a state set not added by the completer is bounded (lemma 6) 
and that the total number of completer operations is ^ n (lemmas 7 and 8). 
Thus the total number of states is ^ n (theorem II). 


Lemma 5 . If G is LR(K), then dmt is a bound on the number of states 

that can be added by a completer operation. 

k 

Proof: If dmt +1 states are added by one completer operation to S. 

then 2 of them must be <p, j+l, fj, a> and <p, j+l, f 2> a> for some 
p, j+l, ci, f| t f 2 . So 3 l such that <p, j, f ( , a> and <p, j, f 2 , a> e 
and g q such that <q, q, i, X . + 1 . . . X . + | < > e S.. Now this state was 
originally added to by the predictor as <q, 0, l, X. + |...X. + ^> so 


X. 


i + l 


• X, 


+k 


«H. (C 
k 


p( j+2) 



Therefore let 6 be a string such that 


C p( j+2) ’ ' * C pp “ ^ X i + I -’^i+k 0 


Let g = f+k+|B| . Let X.,. ,,...X = 6. 

3 1 1 i+k+l g 

Then by applying theorem 3 p-(j+l ) times, we get that contains 
<p, p, f |f ot> and <p, p, f 2 , a>, and a = X ( . . .X g+R . If p = j+l, 
we already know that these states were added by a single completer 
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operation. If C - is -terminal, they were added by the scanner. If 
C - is non-terminal and p > jtl, then they were both added by a single 
completer operation on some state <r, r, h, a> for some r, h. 

If they were both added by the scanner, lemma 3 is violated. If 
they were both added by a single completer, lemma 4 is violated. // 


Lemma 6 . If G is an LR(k) grammar, then dmt k is a bound on the number 
of states in S. not added by the completer. 

Proof: Suppose the number of these states is dmt k +l. Then g p, j, f, £ f a 

such that <p, j, f ! , a> and (p, j, f 2 , a) are in S., but not by the 

comp I eter . If j = 0 , then f = f = i . If C . . . . C . i s a terminal str i nq , 

\ l pi pj 

then f | = f 2 = ' “ j * So one of C p| ...C . must be non-terminal. Let h be 

the largest integer h <_ j such that C ph is non-terminal. We know that 

<p, h, fj, a> and <p, h, f a> s S (where g = i-(j-h)), and these states 

were added by the completer. Choosing the first of these, since it was 

added by the completer 3 Z such that <p, h-l, f , a> e z and 3 q such that 

<q, q, Z, Xg + 1 . . •Xg + | < > e S.. This state was originally added to by 

the predictor as <q, 0, Z, X.^. . . .X. , , >, so 

i+l i+k 


X. 


i + l 


e H, (C J _. , . . .C - a) 
i+k k p(h+l ) pp 


so by the same argument as in lemma 5, 3 an extension X....X such that 

1 9 

S g contains <p, p, f ( , a> and <p, p, f 2 , a > , a = X g+|-*' X g+ k' and +hese 
two states were added either by a single completer operation or by the 
scanner, unless p = h. But this would mean that j = h, and this is 
impossible, since C ph is non-terminal but <p, j, f |# a> and <p, j, a> 

were not added by the completer. 
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|t they were both added by the scanner, lemma 3 is violated. 

I, they were both added by a single completer, lemma 4 is violated. // 

Lemma 7 . If G is LRCk), every completer operation represents a 
different step in the derivation of X r -*V 

Proof: By theorem I, any state <p» P, X i + r" X i+k > 

completer is applicable means that (p, p, f> an i-state of x ,--- x itk - 


D p * C p|-" C PP’ X f +I ‘ ' ' X * 


and R “ x | ••• x f D p X ltl ••• X itk 5 

, y v But since the grammar is LR(k), 

for some extension of X | ...x j+k . 

a step (at I) in the derivation of one extension of x ,--- x l+k '« 4 
step in the derivation of any extension of x r -- x itk ' including V"V 
Furthermore, every such completer operation represents a different 
step in the derivation because states are not duplicated in the same 
state set. So in a different completer operation, either p, f, or i 
must be different, making it a different step in the derivation. // 

Lemma 8 . There is a bound of the form Cn on the length of the leftmost 
derivation of a sentence of length n with respect to an unambiguous 

grammar . 

Pnoof, If any non-termina, N has the property that R* uNS for some 

and A, then there is only one derivation N ^ since the grammar is 



56 


unambiguous. Let z be the length of the longest such derivation for 
all the non-terminals in G. 

In any derivation of a sentence in G, let an occurrence of a 
non-terminal N be a null occurrence if A is eventually derived from 
that occurrence of N i n that derivation. 

Let the width of a sentential form be the number of symbols in it 
which are not null occurrences. Each step in a deri\ rion does one of 
four th i ngs : 

(1) Increases the width of the sentential form. 

(2) Is a step in a null derivation. 

(3) Transforms a non-terminal into a single terminal (plus possibly 
some null occurrences ) . 

(4) Transforms a non-terminal into another non-terminal (plus 
possibly some null occurrences). 

In the leftmost derivation of a sentence, there can be at most u 

occurrences of steps of type (4) before there is a step of type (I) 

or (3) (here we are ignoring type (2)). If there were u+ I steps of 

type (4), then we would have N => N =» a in a non-trivial way as part 

12 

of the derivation for some non-terminal N. But then we could also 

* 

ha 3 shorter derivation by bypassing the second N, N =» a. This 
violates the fact that the grammar is unambiguous. There can be at 
most n type (3) steps since the sentence is of length n. And there 
can be at most n-l type (I) steps for the same reason. So, ignoring 
type (2) steps, there are at most (n+n-l)u steps. 
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Le+ an initial null occurrence of a non-terminal be a null 
occurrence which is'nct an intermediate step in a null derivation. 
Initial null occurrences can only be created by steps of type (I), 


(3), or (4), and at most m of them can be created by each such step. 

So there are at most (n+n-l )um initial null occurrences. So there 
are at most (n+n-l )um different null derivations, and each has at most 
z steps. Therefore (n+n-l )umz is a bound on the number of type (2) 

steps . 

So the total number of steps is bounded by (n+n-l >u(mz+l ) * 2umzn. // 


Theorem II . If G is LR(k), then there is a bound of the form Cn on 
the total number of states constructed. 

Proof: By lemma 5, dmt k is a bound on the number of states not added 
by the completer in any state set. So (nt,)dmt k is a bound on the total 

number not added by the completer. 

By lemma 7, every completer operation represents a different step 
in the derivation of X,...X n for an LR(k) grammar. So the total number 
of completer operations is bounded by the length of a derivation, 
which is the same as the length of a leftmost derivation, or Cn, by 
lenvna 8. By lemma 6, at most dmt k states can be added by a completer 
operation. So the total number of states added by the completer is 

bounded by Cdmt k n. 

Summing these two gives us the result: 


(n+l )dmt k + Cdmt k n ^ Cdmt k n // 
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We will generalize the theorem to be proved to include not only 
LR( k) grammars, but any finite union of them. A union of two grammars 
is just the stra ight-forward way of combining the two grammars to 
obtain a grammar which generates a language which is the union of 
the two original languages. 

Definition . A union U of b grammars G, , — ,G b with roots *,>••• > R b 
is obtained as follows: 

(1) Systematically change the non-terminal symbols of ea<_ 
grammar, so that no non-terminal appears in more than one grammar. 

(2) Let the non-terminals of U be the union of the changed 
non-terminals of G|,...,G^ plus R. 

(3) Let the terminals of U be the union of the terminals of 

G | , • • • t G^ • 

(4) Let the productions of U be the union of the productions of 

G ... G plus the following productions: 

I , * * * 9 b r 

R + Rj 


R~R b 

(5) Let R be the root of U. 

Theorem 12 . There is a bound of the form Cn oh the number of -reps in 
computing RECIG, k) if G is a finite union of LR<k> grammars 
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(write U LR ( ^ 


Proof: Let G be a anion of b grammars. 

The first states added to Sq are 


D 


■0 - - R 

R -+• .R, 


0 


0 


R 




0 


The algorithm then proceeds to process each of the grammar 
para I lei . The b different recognition processes cannot interfere with 
each other in any way because the non-terminals from each grammar are 
distinct and therefore the productions (and thus the states) are all 
distinct. So the bound for such a union grammar is b times the bound 
on an LRCK) grammar. (More generally, the time required fo 
grammar is the sum of the times for the component grammars.) 

The bound for an LR(k) grammar is obtained as follows: 

(a , The total number of states is bounded by Cn. so it takes at 


most Cn steps to scan them a 


The scanner adds one state and the 


mus I vH ' r 

pnedictor adds at at* states for each state scanned. So including 

dmt k to test each state, we get C(dmt k > <«t k ,n steps for these. 

(b , The completer performs at most dmt k steps for each state It 

adds, since it adds each one in only one way because the grammar is 

unambiguous. So it performs at most Cdmt k n steps. // 

Notice that this result and the previous bounded state result do 

not hold for any k. That is, , n this case, we know the algorithm wi 1 1 
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run in time n on a union of LR(k) grammars only if the look-ahead it 

uses is at least as large as k. Grammar RR provides an example of 

what happens if k isn't large enough. We have already seen that it 

is recognized in time n if k - I. But if k « 0, (page 47) we can see 

+ha+ s contains 5 states, S 6 states, and in general S. will contain 

2 

; +4 states up through S n< Thus the total number of states is * n and 
so is the time. The reason for this is as follows: After we have 
scanned x' , we could be finished scanning x's (if i = n) and therefore 
we would need to construct states which represent the whole parse of 
the string as follows: 



In the case that k = I , we look-ahead at the next symbol before we 
construct all these states and we do so only if the next symbol is 
a "-T, that is, we do this only once. But if we don't look-ahead, 
we are forced to construct all these states at each step just in case 

that step one was the last one. 

So with the algorithm as it is, we ^ed the look-ahead to get 
the time n result for all LR(k) grammars. There seems to be a way of 
obtaining the LR( k) time n result using k = 0 by a slight change in 

have not investigated this in detail, 


the algorithm. However, we 



'+ describe it here. Even if this 


so it is only a conjecture, and we won 
mo di t i cation were valid, the look-ahead is still very important in that 

. look-ahead of just I -ill cut down the time used sighif leant, y 
many grammars even if it doesn't produce any ne« theoretical results. 

We now have characterized two classes of grammars which can he 
done |„ time n-bounded state and finite unions of LR<k>. Thus our 
a I gor i thm does in time n a much larger class of grammars than any 
previous one. We don't, however, have a ful, characterization of the 
time n grafts for our algorithm. It is clear that there will be 
some grammars which are right recursive (and therefore not bounded 
state, -hich are not finite unions of LR(k), yet which are simple 
enough to be time n. One possible characterization for the time n 
grammars is that they are all grammars for which the total number of 
states constructed is proportional to n. This has not been investigated. 
Notice that the time n conditions, like the LR(k> condition, is 

not symmetric. That is, if -e implemented a recognizer exactly like 

+ +ha + if went from right to left, then we 
the one in this paper except that it wenx 

would get a different set of grammars to be time n. Grammar RL 
(page 63) is an example of one -hich is time n 2 from left to right, 
as the example run clearly shows. However, it seems clear from the 

structure of the grammar that it runs in time n from r g 

Tne time n grammars seem to be rather large; they seem to include 

most unambiguous grammars. In fact the only unambiguous grammars -e 
nave found -hich are not time n in either direction are certain 
pal | indrome grammars with unmarked centers and variations on them. 



62 


(A language is a pat I indrome if each of its sentences is the same 

reversed.) Grammar 'PAL (page 65) illustrates this. S. and S. + | 

each have i+4 states in them, up to and including S n , so the total 

2 

number of states is ^ n . 

The time n grammars also include a surprising number of quite 
ambiguous grammars. Grammar BK (page 39), we have already seen, 
has unbounded ambiguity, yet it is a time n grammar. 

Even more surprising is that grammar GAP has unbounded direct 
ambiguity and yet is a time n grammar. On page 67 is a run on this 
grammar. One can see that all the state sets after S 2 will also 
contain seven states until a "c" is scanned. Furthermore, when 
recognizing ab n c, S n+2 will be 

B -*• be. n 

B -* bB. n-l 

• . • 

• • 

• • 

B -*■ bB. I 

R **• AB . 0 

4> -*■ R.H 0 

So it contains n states; but since the rest of the state sets are 
of bounded size, GAP is a "time n grammar. 
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Grammar RL 




R -* A|aRb 





A -*• a | Aa 





n m n 

sentences: a a d 

(m > 0, n >_ 1 ) 




REC(RL, a 3 . 

0) : 




c <j> -*■ ,R H 

0 

R .A 

0 / 
0 / 

S 2 

R a.Rb 

A -► a. 

1 / 
1 

R •> .aRb 

0 


A Aa . 

0 

A -*• .a 

0 


R .A 

2 / 

A -*■ . Aa 

0 / 


R -*■ .aRb 

2 




R -*■ A. 

1 

S R -*■ a.Rb 

1 

0 / 


A -»• A. a 

1 

A a . 

0 


R -* A. 

0 

R -*■ .A 

1 / 


A •> A. a 

0 

R -»■ .aRb 

1 


A -*■ .a 

2 

R -*■ A. 

0 


A •> .Aa 

2 / 

A ■* A. a 

0 


R -*■ aR.b 

0 

A -*■ .a 

1 


<J) R 

0 

A -*• .Aa 

1 / 




$ -*■ R. H 

0 






R -*• a . Rb . 2 / 

5 

A -*■ a. 2 

A ■* Aa . I 

A -*• Aa . 0 

R -► .A 3 / 

R ■> .aRb 3 

R -*■ A. 2 

A -► A. a 2 

R -*■ A. I 

A -*■ A. a I 

R -*• A. 0 

A -*• A. a 0 

A -* .a 3 

A -*■ .Aa 3 / 

R ■+ aR . b I 

R -*■ aR. b 0 

<J) R.~ f 0 

S 4 <j» R. -4 0 
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Grammar PAL 


A -* x I xAx 


n , 

sentences: x _ 

I , n odd) 





5 

REC(PAL, x , 

0) : 





l 

0 

s 

A -»• xAx. 

0 


<j> -*■ . A*H 

3 



0 

A .x 

0 


A -*■ x. 

2 



0 


A -*■ x.Ax 

2 


A . xAx 


4> A . 

0 


s i 

A -*■ x. 

0 


A -*■ xA.x 

1 


A -*• x.Ax 

0 


A *► .x 

3 


<j> -*■ A. 

0 


A .xAx 

3 


A -*• .xH 

1 

S . 

A -*• xAx. 

1 


A .xAx 

1 

4 

A •+■ x. 

3 

S 2 

A -*■ x. 

1 


A -*■ x.Ax 

3 


X 

c 

X 

+ 

c 

1 


A -*■ xA.x 

0 


A -*• xA.x 

0 


A •> xA.x 

2 


A -*■ .x 

2 


A -*• .x 

4 


A -> .xAx 


2 


A -*■ .xAx 


4 
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A ■+■ xAx. 0 

5 

A -v xAx. 2 

A -* x. 4 

A -*■ x.Ax 4 

<j> -*■ A.H 0 

A xA.x I 

A -*■ xA.x 3 

A -*■ .x 5 

A •> .xAx 5 

S <t> -*■ aH. 0 
o 



67 


Grammar GAP 


root: R AB 

A -*■ a | Ab 
B -*■ be | bB 

sentences: ab'c ( i I ) 

RFC (GAP ■ ab 3 c, 0) : 

S tb -+■ .RH 0 

R ^ .AB 0 

A -* .a 0 

A -> .Ab 0 

S A -*■ a. 0 

1 

R ■> A. B 0 

A -*■ A.b 0 

B .be I 

B -*■ . bB I 

S A Ab. 0 

2 

B ■+■ b . c I 

B ■+• b.B I 

R ■* A. B 0 

A -*• A. b 0 

B -> .be 2 

B ^ .bB 2 


A -> Ab. 0 

B b .c 2 

B ■+• b.B 2 

R -v A.B 0 

A ^ A.b 0 

B -*■ . be 3 

B •> .bB 3 

A -*■ Ab. 0 

B -*• b.c 3 

B b . B 3 

R A.B 0 

A -*■ A.b 0 

B -*■ .be 4 

B -*■ . bB 4 
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8 -*■ be. 

3 

B -*■ bB. ' 

2 

00 

i 

c r 
CD 

1 

R -*■ AB. 

o 3 

<fi R*H 

0 

<fi - R-j. 

0 
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‘X. THE COMPILED ALGORITHM 

We can recognize a large class of grammars in time n, but for 
practical purposes we are forced to ask, how large is the constant 
coefficient of n? Unfortunately, it turns out that if one uses the 
algorithm as given, the coefficient is likely to be quite large, at 
, eas+ com pared with those of most of the time n algorithms. This is 
understandable in a way, since they are specialized algorithms, 
tailored to specific classes of grammars, and ours is a general 
algorithm which just happens to be time n on many grammars, without 
its being told that they are of any special form. 

But we can expect better performance from our algorithm if we are 
willing to tell it that it has a time n grammar (time n for our algorithm) 
If we know that, we can run the grammar • through a compilation process to 
produce a recognizer for the grammar. Then we can use only the compi led 
recognizer each time we recognize a string. This will cut down the 
coefficient of n to approximately the same magnitude as the other time 
n algorithms. The reason for the cutdown is that the compi I at. on 
process does much of the work once that would otherwise have been done 
with every run of the recognizer. Let's look at an example. If we 
examine an ordinary run on grammar LR (page 71), we can see that 
the states after S, are identical. So we can compile an Algol- 1 ike 

parser for it as follows: 
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S : If X. = x then 

°0 i 

begi n i «- i + l ; go to S ( end 
else Error; 

S • I f X. = x then 
I ' i 

begin i i + l; 9 C I en ^ 

else if X. = H then Halt 
else Error. 

We have combined S, and S 2 because their actions are the same even 

though the state sets are different. 

We won't be as lucky with all time n grammars as with this one. 

We won't always get them to repeat their states so easily, especially 
to repeat them with the same values for the pointers for all length 
input strings. But we do know that in the case of a time n grammar, 
there is a bound on the number of states in any state set.* So i f we 
ignore the pointers, there are only a finite number of distinct state 
sets which can ever appear in the recognition of a string with respect 
to this grammar. Therefore, given a particular grammar, we would like 
to compute all such state sets at compile time, compiling a recognizer 
which works according to the algorithm, but only examines the pointers 
and the input str during the run-time recognition process. For each 


*Th i s isn't really true for right recursive grammars, but 

be in the compi led algorithm. 


it will 
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Grammar LR 


A -*■ x I Ax 

sentences x n (nil) 

REC(A.xx.O) : 

S Q 0 -► .AH 0 

A -*■ .x 0 

A -*■ .Ax 0 

S | A -*■ x. 0 

0 -*■ A.H 0 

A A.x 0 

S 2 A ^ Ax. 0 

0 + A.H 0 

A -*• A.x 0 

S, 0 -►AH. 0 

3 
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compile-time state set we compile a block of code which at run time 
tests the input string and the values of the pointers in order to 
construct a new set of pointers and go on to a new block of code. 

As an example, we show the compile-time state sets for grammar RR 
(page 73, left side). Notice that along with each state in the state 
set, instead of a pointer, we keep a list of all the state sets which 
can be pointed to by a pointer in that position. The code on the right 
side of page 73 is abbreviated code for the Algol code to manipulate 
the pointers and input string. The expansion of this code is on 
pages 74-75, and a sample run is on page 76. We have numbered the 
productions in the grammar, so that these numbers could be used as 
indices for the run time data structures in the compiled code. The 
block of compiled code for each state set is put immediately to the 

right of it. 

The meaning of both the abbreviated code and the compi led Algol 
code is explained in the following full description of the compilation 
algorithm. It is advisable to read this description first and then 
examine the grammar RR compiled code. The run-time data structures for 
which we are compiling code are as follows: 

Run-Time Data Structures 

(1) A three dimensional array F[0 :n+l ,0 :d- I ,0 :m- I ] . We store at 
F[i,p,j] all f such that < p , j , f ,<*> e S . for some a . 

(2) The input string is X[0:n+k+l]. 

(3) A vector S[0:n+1] contains the names of the state sets current at 


each i . 
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Grammar RR 


A 1 x I 2 xA 

sentences: x (n — ^ 


S 0 0 + -AM M S 0 

A -*■ .x M S Q 

A -► .xA M S Q 

S, A -*■ X. M S 0 ' S | 

A x.A M 2 q » S | 

A -*■ .x M S | 

A -► . xA M S, 

S 2 A - xA. M S q , S | 

0 A.M M Sq 


Condi t i on Act i on 

© 1,2 

X (1,0) 

( 2 , 0 ) 


Go To 


S 


I 


1,2 

x (1,0) 

(2,0) S, 

M, I =S , (2,D S 2 

! = s 0 ^°> Q 0 S 3 



S. 0 -► AM 
4 


HALT 
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Comp i I ed 


Code 


<- 0 ; 


F[i ,0,0] 


i; F[i,l,0] - i; F[i,2,0] ^ i; SCil ** S Q ; 


if X[ i + I ] = X +hen 

begin F[i + I,l,l] - F[i,l,0b Fti+1,2,1] - F[i,2,0]; i 
go to S | end e i se 
Error ; 


i + i ; 


s, : F[i,l,0]* i; Ft. , 2 , 0 ] - i; S[l ] * S,; 

if X[ i + I ] = x then 

begin F[i + l,Ul “* F[i,l,0]; F[i + i,2,l] 
go to S | end e I se 
if X[ i + I ] = ^ +hen 


F [ i ,2,0]; 


i + i; 


if S[F[i,l,m = S, then 
t gin F[i ,2,2] **• F[F[i , I , I 3»2, I ]; 


go to 52 e I se 


if S[F[i,l,m = S Q then 

begin F[i,0,l]~ F[F[i , I , ll,0,0]; go to S 3 end 
Error else 
Error; 



S[i ] - S 2 ; 


if X[i+I] = H + hen 
if S[F[i,2,2]] 
begi n F[ i ,2,2] 


= Sj then 

«- F[F[ i ,2,2] ,2, I ] ; go to 


S 2 end else 
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if S[F[i,2,2]] = S Q then 
begin FD>0>l] F[F[i,2,2],0 

Error else 
Error ; 


S 3 : if X. + | > -I then 

begin Ft i ,0,21 - Ft! ,0,1]; go to S 


Error ; 



Halt; 


0]; go to S 3 


. end else 
4 
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Trace of Compi led Run. 


X[i] 


X 

X 

X 

s[i 1 

s o 

s i 

s i 

S ( ,S 2 ,S 2 ,S 3 

F[i,l,0] & • 
F C i ,2,0] 

0 

i 

2 

3 

F[i , 1 , 1 1 & 
F [ i ,2, 1 ] 


0 

I 

2 

F[i,2,2] 




1,0 

F[i ,0,0] 

0 




F[i , 0 , 1 ] 




0 

F [ i ,0,2] 






0 
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At compile time we manipulate objects which we wi II call C-states, 
because they are almost like states in the original algorithm, except 
that they contain a pointer to a collection of states rather than a 
pointer to a place in the input string. Throughout this section, we 

Will refer to C-states as states. 

A state collection is the means by which we identify state sets 

Which are the same except for their pointers. 


Definition: A C-state is a quadruple <p,j,T,«>, where P,J, and a are 

as in an ordinary state and T is a pointer to a state collection. 

ripfinition : A Cat^oM^ is an equivalence Cass of state sets 
under the relation S, * S, if there is a l-l correspondence 
between their states such that < P , , j r T, > a !> a ° d <p 2^2 ,T 2' a 2 

correspond if P| = j | “ and a l °" 2 -' 

nof i n I t i on : where S is a state set, is a pointer to the state 

collection of which S is a member. 

A precise description of the compilation algorithm is on pages 
80-83 The following is an intuitive description: It is a recursive 

procedure which takes as input a state set. It is supposed to compi I. 
the code for that state set and for all state sets which we might go to 

from that state set in recognising any input string. 

,t works as follows: Given a state set S. it computes the pred.ctor 

on S and compi les any code needed to perform this at run time. Then, if 
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there ere no final states in S, it takes each legal terminal x that 
could appear next ip the input string. For each of these, it pretends 
that x was next, computes the scanner under this assumption (compiling 
code for it), and calls itself recursively to compile the new state 
set which results. Thus we are in some sense running the recognizer on 
an possible input strings at compile time. This can't really be done, 
of course, so we must have a termination condition to keep our 
recursive calls on the compilation algorithm from going on indefinitely. 
Before we describe this, however, we wi I I describe ho. the completer 

works. 

If there is a final state in the state set, then for each 
possible look-ahead string a, we first compi le a test to see if the 
next k symbols match a. Then we compute the completer on all final 
states for which a is the look-ahead string (compiling the code for 
this), we delete the final states, replacing them by the states added 
by the completer, and then we call the compilation algorithm 

recursively on the newly created state. 

Notice, however, that we cannot compute the completer in the 
ordinary way. We are at compi le time, so we have no input string and 

no pointer f. Instead we have a pointer to a state col lection. We 

take each of the state sets in this collection one by one and assume 
that it was the state pointed to; then we compute the completer in the 
ordinary way. For example, in grammar RR (page 73), 5, is a state 

collection consisting of two states— one with S Q as the pointer for the 

• c a+ one poi nt we are working with the 
first two states and one with S,. At one poim 

state set consisting of the single state 
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xA. 


ca „ get two different state sets 

In computing the completer on this, we ^ two state sets are 

dependi ng on -Mot of - members of S , 


xA. 4 


3 0 


and 


A -*■ xA. 


■ ters to state collections instead of jest to 
The reason we use dit ion to the algorithm. We 

, H a terminating conditio 

states i s SO we can ^ conditio „ oust have the property 

describe that now. ^ ^ have proce ssed all possible 

that the algorithm te- ^ ^ any string with 

state sets that can be the process of compiling a 

respect to the grammar. Suppose _ „ e encounter another 

state set S land the recursive « ^ any new state setsf The 

^ C0 " "" TtonW if some new state set has been created 

answer is that it can, , f no „ew state sets have been 

since the last time that pro duce only the state sets 

created since that t,me ^ terminate this branch of the 

that the old one produces, 

algorithm now. + four blocks of the 

f lowchart (page 8H. ^ enCOU ntered we mark all states 

“’Tli-try to compile a state which is marked processe, 

unprocessed. Bo 
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that means that no new state sets were encountered since the last time 
i + was comp i I ed . 

No+e that at the end of the completer, we must compute the scanner 
on any states which were not included in the processing done by the 

comp I e+er . 

For each state collection, the compilation algorithm compiles a 
block of code labeled by the name of the state I lection. These 
blocks consist of an action followed by a sequence of tests. Each 
test consists of a condition, action, and go to. The tests are 
implicitly followed by a branch to an error exit if they all fail. In 
the description of the compilation algorithm, "compile under -S" means 
compi le that code into the block of code labeled -S. 

The compilation algorithm we have described here assumes that 
there are no null productions. This problem can be overcome, but we 

choose to ignore it in this description. 

This compilation algorithm has not been either debugged on a 
machine or proven correct, so it may have some small bugs, but the 
genera I idea is not in error. 

The Compilation Algorithm C0MP(G,k) : 

Add <0,0,-*S,-| k > to S. 


Compute COMP I LE ( S ) . 
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FRED I CTOR 

Scan the states of S in order. For each state <p,j,T,a>, if 
j < p and C p(j+|) is non-terminal, then for each q such that 

C p(j+n = D q , and for each 6 e H k (C p ( j+2) 1 ' - C pp a } } ' 3dd 


<q,0,-*S,6> to S and compi le under ->S : 


"q. " 


This is interpreted 


"F[i ,q,0] 


COMPLETER 

For each a such that 3 <p,p,T,a> e S, let < P , , P , , T,a> , . . . , 

p T > be the final states in S with look-ahead string a. 
y r’ v r r 

For each possible combination of a state set S.^ e 6 V 

do the following 3 steps: 


(1) S' - {<P,j,T,S>lj < p and a e H k (C ( j+| } • , c pp 

a r 

(2) For each state <q . ,1 . ,U,B> e S. such that C q (jL , ^ D p ’ 

J J j J J J 

add <q . +1 »U,6> to S' (j = \ , . . . ,r) . 

J J 

(3) Compute COMPI IE (S^) 

Compi le under -*-S : 

"a , p | = T j (q,,*,) [for each q,, l, as in (2)1 

; T (q ' A ) S' " [for each q r ,* r as in (2)] 

r r r 01 

This is interpreted 

"if X[i+l]...X[i+k] = a A 


S[F[i ,p, ,P ! 11 = ^T| A ... A S [ F [ i , p p , P r ] 1 ""T then 

begin F[i ,q, ,1 , + l ] F[F[ i , p , , P , Lq , > l \ ^ ^ for each q r £ l^ 


F[i 


,q r ,t r+ l] - F[F[i,P r ,P r l,q r .» r ] [f ° r eaCh W 1 



St i 1 «- -*-5' 

a 

Go To S * 

end” 

Let S" - {states sis £ for any of the above a} 
Compute SCANNER (S'). 


SCANNER (U) 

For each x such that 3 <p,j,T,a> ® U such that 

x = C p(j+!)' 

U' <- (<p, j+l,T,a>l<P, j,T,a> e U and c p( j+|) 

Compi le under -*-S : 

"x (P|.j,) 


x } 




where 


= (<f,j 


3 ct,T such that <p, j+l > 


The above compi led code means 

"if X[i+I ] = x then 

begin F[ i + 1 , P | > j | + F[i>P|>j|} 

F E i+l ’ P m’ J m +l] " F[i,P m ,j m ] 
i i + ! 

S [ i ] «- -KJ ' 

Go To U' 


end" 


,ot> e U' } 


Compute COMPILE (S') 
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lf ue examine the grammar RR example .page 73, in detail h .Ml 
find that some of the run-time pointer manipulation which gets oompiled 
is unneoessary. In particular the four circled parts of the 
abbreviated code are unnecessary. Extraneous code like this may 
appear because the pointer is set but never tested in a completer 
operation, or if it is tested, there may be only one possible value 

for i+> so the test is meaningless. 

any case, we can prune away al I extraneous code by making an 

analysis of the oompi led code. This takes the form of working backwards 
from all meaningful pointer tests and marking all the code which leads 
up to them, we could then delete all unmarked code as being useless. 

We expect that such a pruning procedure would significantly improve the 
run-time speed of the oompi led recognizer. For many grammars we could 
also extend this pruning procedure to delete state sets which accomplish 
nothing (after pruning, and to combine any two state sets whose actions 
are the same. Grammar LR2 (page 125, provides a good example of 

pruning. 

Now we investigate just what properties this compilation algorithm 
has. If the algorithm does terminate, it's clear that no new state 
sets could have been created, so we oo have a valid recognizer for the 
grammar. Can we also say that the grammar was a time n grammar? 
Unfortunately, not quite. What we do know is that there 
f ,nite number of state sets which can be created by the grammar and 



therefore there is a hound on the size of any of the™. But these state 
sets are not quite' the same as those of the original algorithm. The 

difference comes in the way we treat the completer. In the original 

algorithm, we compute the completer on all the states in the state set, 
including the. which may have just been added by the completer, until 
there are no more states to which the completer i s app I icab le. But in 
the compilation algorithm, we apply the completer once, deleting the 
. old final states and substituting the new ones to form a new state set. 
Then if the completer is applicable again, we apply It to 

state set, again deleting the final states after it is applied. So in 

some cases where we might end up with mi states in S. according to the 
Old algorithm (because of the accretion of old final states), we can 
get at most a bounded amount by the compi lation algorithm because of 
the deletion of final states. Grammar RR is a perfect example of this. 

the original run (page 47), S 3 has 7 states (and S„ would have had 
„44 states) but in the compi led run (page 73), no state set has more 

than 4 states in it. 

From this insight we should be able to come up with a definition 
of what grammars can be compiled by this algorithm. We will not try 
to make it independent of the algorithm. 


Definition : A grammar G is compi I able with look-ahead k, COsi, if 3 

a bound on the number of states in any state set not added by the 
completer, and 3 a bound on the number of states added by any 
single completer operation. 
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+ha+ there is a bound on the number 
This is equivalent to saying that 

, , bv the compi lation algorithm. Now 

w e+ate set qenerated by tne cmn 

of states i n any state , t i s not BS( I) , but 

DO : c pin even thougn u 
by this definition, grammar RR 

Y . n k . , also provides an example of 

;+ is LR(l) and time n using k 

+ . ,+ is C(0), but as we 

. . h is compilable bu^ not time n. 

Qnmpthinq which is comp ^ 

• .* . +nkes time n when run with 

have seen before (page 47), it def.n. e Y 

" = O so , conclusion, »e nave tout Casses ot grammars having tne 
fol lowi ng relations: 

U LR(k) ctime n 
bounded state ctime n 

time n a . compi I able 

, th u , R(k) nor bounded state is included in the other. 

3 " d " a teseihCusionsareproper. - have a, ready shown 

FUr+her " 0re ’ t that there are time n .radars and bounded state 

a „ these facts excep This wili be shown in Section 

grammars which are not U LR(k). 

y -i 4 --ran a I nor i +hm terminates, the 

, +hat if the compilation algoriTnm 

So we know that it . ,_ h i e 

... bl . lt is also obvious that tor any non-comp, lab.e 

grammar is compilable. there will be an 

grammars, the algorithm won't terminate, becau 

. . dif ferent state sets to be processed, 

infinite number ■ ^ ^ ^ „ „ .fortunate, y 

terminating on i ndef i ni tely on a compi lable 

also possible for the algorithm to go 

This can happen in the following way: 

9 + ad that if the algorithm terminates. 

Not i ce that we have guaranteed that 

a „ the state sets that oouid be needed wii, have been created. 



67 


not guarantee that onj* those needed would be created. In tact this is 
no+ „ eC essarily sc- When we perform the completer on a state set S, we 
pick up all the state sets in the state collection which is pointed to 
by some final state s in S, not just the one state set which was 
current when that s was created originally. So in using all these 
state sets, we may create a new state set which couldn't have been 
created by any run of the original algorithm. And, of course, if we 
can produce- one unneeded state set, the processing of it may produce 
an infinite number of unneeded ones, where the number of needed ones 

was finite. 

Given this, one wonders why we use state collections at all. Why 
not just perform the completer on the one state set which was valid 
when s was created and avoid this trouble? We could, but close 
inspection of the algorithm wi 1 1 reveal that this would invalidate the 

terminating condition we are using. 

So in conclusion, if the compilation algorithm terminates, the 

grammar is compilable. If it doesn't terminate, the grammar may or 
may not be compi lable. In Section XII we show that one cannot expect 
to do better than this in deciding whether or not a grammar is 
comp i lable. 

if the compi I at ion algorithm doesn't always terminate, even on 
compilable grafts, how can we use it in a practical sense to compile 
a recognizer for a grammar? One solution is as follows: We run the 

algorithm with a built-in limit on how far it can go. This could be a 
time limit, a limit on the size of any state set constructed, a limit 

sets constructed, etc. With some 


on the total number of state 
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experience we could obtain reasonabie estimates for these limits as a 

function of the size of the gram^r . Then we run the compi lation 

if ;+ compiles the grammar, we 
algorithm with the limiting condition. If it comp 

are through. If it terminates because of the limiting condition, - 
print out the state sets constructed so far and examine them. In most 
case s, we should be able to tell by examination whether or not it 
appears that the algorithm will not terminate-this might be because a 
sta te set appears to be growing in size in some uniform way and there 
appears to be no reason to limit its growth. If it appears that the 
aigorithm won't terminate, we can give up on this grammar deciding tha 
i+ , s not compilable (or it might be one of those few compilable 
grammars on which the algorithm doesn't terminate,. If it appears 
that the algorithm will terminate, we run it again with a limiting 

. +K* nrncess This should not be a serious 
condition and repeat the process. 

. ,„ s at least, because it seems likely that all 
problem, in compilers at leasT, 

programming language grammars will be easily compilable. 
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XI . AN EXAMPLE 


section IX we claim that there are time n grammars »hich are 
not finite anions of LRdO and we claim that the time n grammars are 
very large. In Section X we also claim that U LR(k) does not include 
bounded state. In this section we want to give these claims some 
validity by producing grammar N, which is obviously not U LRdO, but 
even more strongly, L(N> is not a finite union of deterministic 
languages (those languages generated by LRdO grammars). We run the 
compilation algorithm on grammar N and it terminates. So N is 
compilable. Further, N is not right recursive, so N must be bounded 

state as well, and certainly a time n grammar. 

L(N) was obtained from [Ul 65], pages 22-25, where it is proved 

that it is not a finite union of deterministic languages. Grammar N, 

L(N) , and the compilation with k = I are on pages 90-96. This should 

also serve as a more interesting example of the application of the 

compilation algorithm. We have used some abbreviations: "4ac" in the 

condition column means that the next symbol can be any of these. 

"8=S " means that 8 can point back to either S, or S 2< 

I ,2 
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Grammar N 


A -► 8 aAbl 9 ab 
C -► 6 aCbb 1 7 abb 
D - 3 aI 4 Ac1 3 Ccc 
r -> 'a| 2 RD 


L (A) 

= 

n, n 
a b 

(n > 

i) 

L (C) 

= 

n, 2n 
a b 

(n 

1 1) 

L (D) 

= 

n.n 
a b 

(n j 

>_ 1) 


u 

n, n 
a b c 

(n 

1 1) 


u 

n, 2n 
a b 

cc 

(n i 1) 


L(R) = L(D)* 


S Q 0 -*■ .R-» 


R -*■ . Ha 

R -> .RD 4 a 

0 -*• R. H H 

R ->R.D H a 

D -*• .A Ha 

D -»■ .A Ha 

D -*■ .Ccc H a 


A 

A 


.ab 




. aAb H ac S, 


H ac S 


0 


C ->-.aCbb c Sq 
C -> c S n 


Condition 


Action Go To 

0,1, 2, 3,4, 5, 6, 7, 8, 9 

( 0 , 0 ) 

( 2 , 0 ) 

( 6 , 0 ) 

(7.0) 

(8.0) 

(9,0) S, 

( 0 , 1 ) S 33 
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P.nndi tion 


A -*■ a.Ab 

^ ac 

S 0' S 26 

A -*■ a.b 

H ac 

S 0 ,S 26 

C -*■ a. Cbb 

c 

S 0 ,S 26 

C "*■ a.bb 

c 

S 0 ,S 26 

A - .aAb 

b 

S l 

A -*• .ab 

b 

S l 

C -*■ .aCbb 

b 

S l 

C .abb 

b 

S l 


Acti on Go To 

6 , 7 , 8, 9 


( 6 , 0 ) 

(7.0) 

( 8 . 0 ) 

(9,0) S 2 


(9.1) 

(7.1) S 3I 


A a.Ab b ^ I '^2 

A -*• a.b b S j /S 2 

C a .Cbb b ^ 1 '^2 

C -*■ a.bb b s | , S 2 

A .aAb b ^ 

A .ab 6 S 2 

C -*■ .aCbb b S 2 

C -*■ .abb b S 2 


a 


b 


6 , 7 , 8, 9 


( 6 , 0 ) 

(7.0) 

(8.0) 

(9,0) S 2 


(7.1) 

(9.1) S 3 


Sj A -*• ab. 

C ■* ab.b 


b 

b 


S 2 ,S I 


S 2' S I 


b ,9 - S2 
b,9=S, 


(8,1) S 4 

( 8 , 1 ) S 6 


$ 4 A aA.b b 

C ab . b b 


S 2- S l 

S 2> S I 


( 8 , 2 ) 

(7,2) S 5 
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- 

Condition 

Action 

Go To 

A -> aAb. b 

S 2 ,S I 




C -*■ abb . b 

S 2 ,S I 

b ,$-$2 

(8, 1 ) 




7=S 2 

(6,1) 

S 8 



b ,8=S | 

(8,1) 




7=5 1 

(6, i) 

S 29 

A aA.b H ac 

S 0 ,S 26 

b 

(8,2) 


C -*• ab.b b 

S 2’ S I 


(7,2) 

S 7 

A aAb. H ac 

S 0' S 26 




C -*■ abb. b 

S 2' S 1 

M ac,8=So>S2g 

(3,0) 





(4,0) 

S 27 



b , 7 -S 2 

(6,1) 

S|6 



b,7=S , 

(6,1) 

S I 9 

A -*• aA.b b 

S 2 ,S I 

b 

(6,2) 


C -*■ aC.bb b 

S 2' S I 


(8,2) 

S 9 

A "*■ aAb. b 

S 2' S I 

b ,8=S2 

(8,1) 

S I0 

C aCb.b b 

S 2' S 1 

b,8= c 

(8,1) 

S 14 


s 10 A aA. b S 2' S I 

S 2 ,S I 


C aCb.b b 


(8,3) 


S 
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Condition 

Action 

Go To 

A -*■ aAb. b 

S 2' S 1 

b,8=5 2 

(8,1) 


1 

C -*■ aCbb. b 

S 2' S I 

6=S 2 

(6, 1 ) 

S 8 



b ,8=S | 

(8,1) 




6=5 ( 

(6,1) 

S 12 

A -*■ aA.b H ac 

12 

S 0 ,S 26 

b 

(8,2) 


S 0' S 26 


(6,2) 

S 1 3 

C -»• aC. bb c 



; A -*■ aAb. H ac 

C aCb.b c 

S 0 ,S 26 




S 0' S 26 

“1 ac,8=SQ,S 2 g • 

(3,0) 





(4,0) 

S 27 



b 

(6,3) 

S 2I 

S A -*■ aA.b H ac 

14 

S 0' S 26 

b 

(8,2) 


p r 


(6,3) 

S 1 5 

C -*■ aCb.b b 

S 2 ,S| 



S A -*• aAb. ^ ac S Q ,S 26 

b ,6=S 2 

(6,1) 

S 1 6 

1 J 

C ■+ aCbb. b 

S 2 ,S I 

b,6=S ( 

(6,1) 

S 1 9 



H ac,8=S 0 ,S 26 

(3,0) 





(4,0) 

S 27 

S C aC . bb b 

5 16 

s,,s 2 

b 

(8,2) 

S I7 


C C 

K 

(8,3) 

S |8 

S _ C -*• aCb.b b 
17 

b | ' d 2 

U 
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Condition 

Act i on 

Go To 

S 1 8 

C -*■ aCbb • 

b S r S 2 

b , 8-S 2 
b ,8=S j 

(8, 1 ) 
(8, 1 ) 

S ,6 
S 1 9 

S t9 

C aC.bb 

c S 0' S 26 

b 

(6,2) 

S 20 

o 

CM 

CO 

C -> aCb.b 

c S 0’ S 26 

b 

< 

S 2I 

S 2I 

C -v aCbb. 

c S o' S 26 

c,6=S 0 ,S 26 

(5,0) 

S 22 

S 22 

D C.cc 

"* a S 0' S 26 

c 

(5,1 ) 

S 23 

S 23 

D -v Cc.c 

4 a S 0 ,S 26 

c 

(5,2) 

S 24 

*3* 

CM 

CO 

D -* Ccc. 

H a S 0 ,S 26 

H a ' 5;=S 0’ S 26 

(2,1 ) 

S 25 

S 25 

R + RD. 

Ha S Q 

H a,2=SQ 

(2,0) 

(0,0) 

S 26 

S 26 

0 -> R.H 
R -► R.D 

“• s o 

Ha S Q 

a 

3, 4, 5, 6, 7, 8, 9 
(6,0) 



0 -> .A 

H a S 26 


(7,0) 



D -Ac 

a S 26 


(8,0) 



D Ccc 

-1 a S 26 


(9,0) 

S l 


A -► . aAb 

H ac S 26 
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Condition 


A -*• . ab -i ac 

5 26 

-1 

C ■> .aCbb c 

S 26 


C -*■ .abb c 

S 26 


, D A. “1 a 

27 

S 0’ S 26 


0 -*■ A.c -i a 

S 0' S 26 

4 a,3=S Q ,S 26 
c 

c 0 "*■ Ac. a 

b 28 

S 0' S 26 

-1 a > 4=S 0' S 26 . 

c A aA.b -i ac 

b 29 

S 0’ S 26 

b 

C aC.bb c 

S 0' S 26 


S 30 A ■*" aAb ‘ 3C S 0’ S 26 


C -*■ aCb.b c 

S 0’ S 26 

-1 ac,8=S Q ,S 26 


b 


Act i on 

( 0 , 1 ) 


( 2 , 1 ) 

(4, 1 ) 

( 2,1 ) 

( 8 , 2 ) 

( 6 , 2 ) 


(3.0) 

(4.0) 
(6,3) 


Go To 


S 


33 


25 

’28 

25 

30 



S 3I A 


ab . 


-I ac 


S 0’ S 26 


H ac > 9 _s o ,S 26 


b 


(3.0) 

(4.0) 
(7,2) 



C -* ab .b c 


S 0' S 26 
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Condition 


Act i on 


Go To 


S 32 C - abb. c _ 


o,7=S n ,S, 


S 33 0 Rt . -i S Q 
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.XI 


SOME UNDECIDABILITY RESULTS* 


Since we have a set of compilable grammars for which a fast 
recognizer can be compiled, we would like to know (I) Can it be deter- 
mined for all grammars whether or not they are compilable? and (2) If 
not, can we at least have an algorithm that terminates if a grammar is 
compilable and not otherwise? In other words, are the compilable 
grammars decidable, or at least enumerable. In this section we show 

that they are neither. 

We will use the famous Post correspondence problem [Po 47] to get 
our resu Its. 


Y _ , v ) . The Post correspondence 

Def_i nition : Let X - (x | > • • • > x w ) ' I'"*’ w 

problem P(X,Y) is solvable iff 3 a sequence i, 


i , . . . i (y i D 

i , y — 


(1 ^ I I — 


< w) such that x. . . .x. - Y j • • - Yj • 

1 1 v i y 


This problem was shown by Post to be undecidable. It is used in 

[BPS 64] to show that determining whether the intersection of 

context-free grants is empty is also an undecidable guestion. We use 

that same construction to show that this guestion is also undecidable 

t(, : c construction uses the following set of 
for compilable grammars. This construction 

comp i lab 1 6 grammars. 


*The proofs in this section were suggested by R. W. Floyd. 
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Definition : The grammar G(X) is as follows 

root: R - ab ’ cx .-I ab ' Rx . (I 1 i 1 «) 

sentences: ab ^...ab c x. ...x. (y L ' > 

I Y 


1 ' 


. , i < w) 

y - 


Def i n i t i on : 0 is the empty set. 


Theorem 13: The problem, "Is U "3 , > n L<G 2 ) empty?" is undecidable for 


G, and compilable grammars. 


Proof: G(X) is LR(O) for any w-tuple X as follows: All sentential 

forms are of the form 


ab ' Y . . .ab '(c or R}a 


. < • : < w> a. The handle ends somewhere in 

For some y 1 U * ' 

The exact place however is determined unambiguously by i, (which is 
obtained by simple counting, since it takes on at most w values), 
the handle is determined without looking to its right at all. 
the grammar is LR(O) and consequently compilable. 

PCX, Y) is solvable^KG(X)) fl L(G(Y» / 0 as follows: If P(X,Y) 

is solvable, then 3 i,...I y such that x.^.-x.^ = Y i( --- Y iy * Bu+ +he " 
ab' Y . . .ab 'c x. ...x. (which e L(G(X))) 

' I y 


= ab 'y ab ' 'ey. ...y. (which e L(GCY)))- So the intersection is 
'l 'y 


non-empty. If L(GCX) ) n L(G(Y) ) is non-empty, let 
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ab 


Y _ ,ab 1 ca be the common sentence. ” "i , i. 


. Then a = x. . - .x. and 


a = y . • ■ -Y 


_ . y . , so P(X,Y) i s solvable. 


So if the intersection problem were decidable for compilable 
grammars, we could decide PfX.Y, as follows: Decide whether or not 


L(G(X) ) n L(G(Y) ) is empty, 


f it is, then P (X, Y) is unsolvable and 


vice versa. 

But the Post correspondence problem is undeci dab I e. // 

now, using Theorem 13, we can prove that the compilable grammars 
are undec i dab l e . 

Theorem 14 : The problem, "Is G compilable?" is undecidable for any 

grammar G. 

Proof: Let G, and G 2 be compilable grammars, with roots R, and R 2 - 

Consider the grammar B(G,,G 2 >: 


root : 


R 

A, 


A| A 2 

A I A j c Rj 


where c l G . ,G- 


a 2 -*■ a!a 2 c r 2 


plus the productions for G, and G 2< 


sentences: cci| . . .ca. c B| • • ,c ® s 


where « G, and 6, 6, <n.*± 0 ' 
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(a) lf y e UG,) tVL(G 2 ), then B(G,,G 2 > will have the same 
compi lability properties as grammar BP (page 40) with "cy" replacing 
"a" in grammar BP. And we observe from the run of BP, that in 

S ( i > I ) we have the states 

i — 

N -► Na. 0 
N -*■ Na. I 


N -*■ Na. i-l 


assuring us that the grammar is not compilable. 

<„) On the other hand, if LCG,) n L(G 2 > - then since G, and G 2 
are compi lable and c forms a convenient marker between their sentences, 
there can be no trouble compiling a recognizer for B(G,,G 2 >. The 
compilation algorithm will compi le a recogni zer which initially looks 

for either a G, or a G 2 between the c's and as scon as it gets the 
first G 2 , it looks only for G^s. So B(G, ,G 2 > is compilable. 

Therefore, L(G, ) n L(G 2 > - S> B(G, ,G 2 > is compilable, and if the 
compilable question were decidable, we could decide the intersection 
question for compilable grammars. But this is impossible by Theorem 13.// 


We can also prove that the BOA question is undecidable using this 
same grammar. 

Now that we know that the compilable -ammars are undecidable, it 
is reasonable to ask whether or not the.. -■ at least enumerable. We 
use the Post correspondence problem again ro show that they are not. 



Theorem 15: 


The set of all solvable Post correspondence problems is 


enumerable. 

Proof: The fol lowing algorithm enumerates a I I solvable Post correspondence 

problems over a vocabulary V: 

Enumerate all triples of integers <w,«.,y>. On each one perform 
the fol lowi ng: 

Enumerate all possible pairs of w-tuples <X,Y> = < (x j , . . . ,x^) , 

/ v )> of strings over V of length <_ l. There are a finite number 

y I ' ' ’ w 

of them. For each of these, enumerate all possible y-tuples of 

integers (i such that I 1 i , i y ± "• There are a finite 

number of these. If x. ...x = y ...y , then output P(X,Y) as a 

•i 'y i y 

solvable problem.// 

Theorem 16 : The set of all compilable grammars is not enumerable. 

Proof: If we could enumerate all compilable grammars, we would be 

able to enumerate the unsolvable Post correspondence problems (PCP) as 
fo I lows : 

For each Post correspondence problem P(X,Y), using the proofs of 
Theorems 13 and 14, we know that 3 compilable grammars G(X) and G(Y) 
such, that P(X, Y) is unsolvab le<^B(G(X) ,G(Y) ) is compilable. 

So enumerate the compilable grammars. As each compilable grammar 
appears in the enumeration, we test whether it is of the form B(G(X),G(Y)) 
for some X and Y. If so, we output P(X,Y) as an unsolvable problem. 
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, . | ppp's are also enumerable, so the 

But by Theorem 15, the solvable PCP 

PCR is decidable. This is impossible by [Po47].// 

Since the compilable graders are not enamerable, » *nc that we 
cannot improve much on the compilation algorithm that we have. We 
certainly can’t get it to terminate even on a I 1 compilable gramm 

• i I _ i w : f thev were enumerable, 
for that would be possible nly if they 
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XIII. SPACE 


Our algorithm provides no new results in the way of upper bounds 
on the space required for recognition. The best result in that 
direction [LHS 65] is (log n> 2 , far better than we can do. But on the 
other hand, this result is only obtained using an extremely slow 
algorithm. When compared with other fast algorithms, our space results 

show up quite well. 

Theorem 17 : There is a bound of the form Cn 2 t 0(n) on the number of 

registers used in computing REC(G,x , , . • • >* n > k) for any k and any 
grammar G. 

Proof: dmt k i is a bound on the number of states in any state set 
5. . The 3 representations we have for each state set (2,3,4 under 
implementation, page 26) each require an amount of storage proportional 
to the number of states in the state set. The storage for the grammar 
is a constant, and that for the input string is n. So summing the 
storage over all the state sets: 


n 


n+l 
+ l 


i = l 


dmt i 



So, in general, the algorithm requires space n . This 
to Cocke's and Kasami's algorithms, which also require n . 
has one big advantage over Cocke's in that the n 2 is only an 
bound for ours. Cocke's algorithm requires n 2 all the time. 


i s comparab 
However, it 
upper 
while our 


e 


algorithm very 


often takes only space n. 
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In fact one may have noticed that i n the s e sets that 
longer current, many of the states are no longer useful, and in fact 
are not accessible at all. The only way that non-current states can 
be accessed is by the completer, and many of them are either of the 
wrong form for the completer to access them, or the states which might 
have caused them to be accessed are no longer current. 

Thus we can introduce a "garbage collection" procedure, [Me 62] 
.page 42, which can be evoked at any time, which will release the 
storage for those states which are no longer needed. We will specify 
this procedure only as far as formally defining those states which are 
still needed at point i in the scan. These are those states for which 
there exists a sequence of completer operations which could lead to 

them from a state in S. . 


Definition: A 


states s 


state s = <p,j,f/»> U needed at .1 if 3 a sequence of 
,s,,...,s„ ( H - <P t ,j t ,f t .V 1 ' 0 m) 5UCh 


O’H 

that s Q e S ,, s m = s, and s* e S f (4 _, ) 
(1 <_ i <_ m) . 


and C / t i | \ " 

Pj>J +1) P(ft-l) 


Under the assumption that this garbage collector is at work, what 
can we say about the Cass of space n grafts? ^ey natural, y include 
the time n grammars, for these certainly don’t create more than a total 
O, „ states in any recognition run. However, because of the garbage 
collector, quite a few time n 2 grammars will be space n also. We have 
not been able to characterize them in any meaningful way independent 



of the algorithm. However, they include all grammars for which there 
is a bound on the number of "needed" states in any state set other than 
the current one. 

2 

Grammar PAL (page 65) i s an example of a time n grammar which is 
space n. In examining the run with grammar PAL, we note that in each 
state set, there is only one state, 

A -+• x . Ax i - I inSj 

which can be needed when this state set is no longer current. So when 
the garbage collector is finished, we will have left the states in the 
current state set plus one in each of the previous state sets— 

2 

proportional to n in all. Grammar RL (page 63) is another time n but 
space n grammar. There are 3 needed states in each non-current state 
set for this grammar (we have checked J them). 

So the space n grammars include the time n grammars plus some 
more— possibly all unambiguous grammars, but certainly most of them 
from a practical point of few. 
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XIV. THE PARSER 


we ha ve no. investigated the algorithm thoroughly as a recognizer. 
But .e are really interested in parsing for practical purposes. In 
this section we show ho. to convert the algorithm into a parser, and 
.e investigate what different properties it has in this form. 

The recognizer can be made into a parser in the fol lowing way: 


Each time we p 


erform the completer operation adding a state 


E aD.B g 


(ignoring look-ahead) 


we construct a pointer from that instance of D i n that siate 


state 


D -*■ Y. 


which caused us to do the operation. This indicates that D was 
parsed as y. In case that D is ambiguous there will be a set of 
pointers from it, one for each completer operation which caused 

E aD.B g 

to be added to the particular state set. Each symbo, in y will also 
have pointers from it (unless it is terminal) and so on, thus repre- 

senting the derivation tree for • 

In this way, when we reach t minating 


0 -*» R-l . 0 
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have the parse tree for the sentence hanging from R if it is 
unambiguous, and otherwise we will have a factored representation of 
all possible parse trees. 

The following is a more precise description of this parsing 
a I gor i thm. 

nef i ni t i on : A parse tree is a directed graph with parse states at the 
nodes. A parse state is a 5-tuple <p, j,f,a,B>, where p,j,f and a 
are as in a state and B is a seguence consisting of terminal 
symbols and sets of pointers to other parse states. The elements 
of the sequence are B,...B j . I f C ph i s term! nal 1 1 < h <_ 


+hen B h = C ph‘ 


If c h is non-terminal, then is a set of 


pointers to other parse states. Each of these pointers represents 
a different possible derivation proceeding from C ph ; specifically, 
if a parse state pointed to from a parse state in S. is 
<q,q,t,S,B'», then D q = C ph and the production C ph - C q ,...C- 

is first in some derivation of \ + ['***i from ^ph' ' n 

derivation of the entire input string. The root of the parse tree 

is the parse state <0,2,0,-! ,&> e S n+ |. 


The Parser 

Add the following to the recognition algorithm: 

(I) m the predictor, let the parse state to be added be 
<q,0,i,M>. ' n +he completer and scanner, let the first 4 or j 

state be carried along to the newly created 


elements of B of the parse 



parse 


i i r be a pointer to < p,P^^ a ;B>* 

state. In the completer, let B^ +| d p 


In the scanner, let B^ +) X. +) 


(2, When a parse state <p, j.f ,°,B, ■ • -BP i s added to a state set, 
it the set already contains a parse state <p, j,f ,»,Bj . . .Bl> , then we 
replace that state with 

<o.i.f.a,CB, U B!>...<B, U B'P 


implementation 

We store the sets of pointers B. as 
lemma I, we can get a pointer into a set 
unite the lists (in (2) above) by joining 
of another. This takes a fixed amount of 
of the l i sts . 


linked lists. By the proof of 

B in only one way, so we can 
i 

the tail of one to the head 
time independent of the size 


Now we ask 
the recognizer? 

for the general 

J .. 2 
to the time n 


, does the parser have any different time bounds than 
The answer is no. The following theorem proves this 
case, but it is clear from the proof that it applies 
and n results also. 


theorem 18 : There is a 

required to parse X|...X 


bound of the form Cn 3 + 0<n 2 > on the time 
with respect to any grammar G. 


Proof : 
occurs 
exi sts 


The only difference between recognition (Theorem 5) and parsing 

adding a parse state which already 


in the completer when we are 
in the state set. This may 


take up to j steps to unite the 



I j s + s B| . . . Bj . But j is bounded by m. 
exponent of m in the final bound.// 


This increases by I the 


The parser does 
recognizer. This is 

parse trees. We can 

3 

• that the time is n . 


have different space requirements than the 
because space is required to store the factored 
derive an n 3 space bound, however, by observing 


Theorem 19 : There i s a 

required to parse X ( . . .X 


bound of the form Cn 3 + 0(n 2 ) on the space 

with respect to any grammar G. 
n 


Proof; it takes one operation to use up one storage location, so we 
get this result by Theorem 18.// 

This reS ult indicates that we- are able to store in space n 3 (in a 
factored torn) all the parse trees for any grammar and string, even 
ones which have an exponential number of parses (grammar UBDA, page 42, 
is such a grammar). Even more remarkable, our parser automatically 
builds a factored representation of even an infinite number of p 
Grammar INF (page 1 10 ) I I lustrates this. We have circled the last 5 
states so they can be pointed to. Notice that R -*• R.A 0 i s not part 
ot the final parse tree (since it is not a final state) even though it 

was important in getting the parse tree formed. 

We also notice that for BDA grammars, the space results for the 
parser are the same as those of the recognizer. This is because a 
state can be added in at most a bounded number of ways by the completer, 
and therefore the additional space required for its parses will be 



3r X INF - 

r -*■ a Ira 
a -* xU 

Sentences: x° (n 1 0) 

PARSE (INF. .01 : 

5 0 -*■ .FH 0 

R -*• .A 0 

R -*■ .RA 0 

A ■> .x 0 
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bounded. Thus the parsing space for BDA grammars is n and the space n 

grammars are the same for the parser and recognizer. 

Vie have not specified hoe to convert the compi led version of the 

. , . +0 _ Darser bu + this is done in almost exactly the same 

algorithm into a parser , dui 

way as for the original algorithm. 

The algorithm we have described in this section provides a way of 

constructing the parse tree as the analysis is done. An alternative 
is to run the algorithm through as a recognizer, and then reconstruct 
each parse from the state sets afterwards. This can be done in a 


strai ghtforward manner. 



XV. TWO MODELS 


In this section »e describe in more detai I the random access model 
of a computer that we have been using implicitly throughout this paper. 
We also discuss the possible use of a Turing machine model and the 
results obtained under this different formalism. 


ThP Random Amass Machine (ram) 

This model has an unbounded number of .registers (counters) each 
of which may contain any non-negative integer. These registers are 
named (addressed, by successive non-negative integers. The primitive 
operations which are allowed on these registers are as follows: 

(I) Store 0 or the contents of one register into another. 
a) Test the contents of one register against 0 or against the 
contents of another register. 

(3) Add I or subtract I from the contents of a register (taking 

0 - I =0). 

The control for this model is a normal finite state device, me most 
important property of this n*chine is that in the above three operations, 
the register R to be operated on may be specified in two ways: 

(1) R is the register whose address is n (register n). 

(2) R is the register whose address is the contents of regi 

This second mode (sometimes called indirect addressing, is what gives 
our model the random access property. The time is measured by the 



^ortnrned and the space is measured by 
number of primitive operations performed, 

the number of registers used in any of tnese operations. 

We will not specify thi s model i n any more detai I . 

from the above how an appropriate formalism could be developed. It is 

our opinion that this model represents most accurately the properties 

of real computers which are relevant to syntax analysis. 


The Bounded Activ ity Machine 

Another model, which is frequently used in studies of computational 
complexity, is a bounded activity machine (bam,. This is a Turing 
machine, which has a fixed number of tapes which may be multi-dimensiona, 
and a fixed number of heads. It is a weaker model than the random 

access machine in that any algorithm can be done at least as fast on a 

. „n+ +r-i ip This is because a random 

ram as on a bam. The converse is not true. 

access machine can access anything in its memory in one step (if 

the address,, while a bam must move one of its heads to the appropriate 

position first. This could take proportional to n steps in seme cases. 

Since the bam is a frequently used model, it is reasonable to ask 

if we can obtain the same results for our algorithm if it is implemented 

on a bam. We are unable to obtain any of our results by a "direct" 

encoding of the algorithm onto a bam. AH the times get increased by ^ 

at least a factor of n in this way. We have been able to obtain the n 

.. . | ypd head Turing machine by making a 

result on a two-dimensional, fixed neaa iur a 

small change in the algorithm to fit the bam structure, but we have 
been unable to obtain the n 2 or n results at all on a bam. 



This coincides- quite closely with the published results. Cocke's 
n 3 a , 90ri thm does work on a bam. Kasami 's n 2 algorithm tor -ambiguous 
grammars works on a ram only. Knuth's al gor i thm i s ab le to compile a 
push down automata (certainly a bam, for LROO grammars, but this is 
not a general context-free algorithm. So our algorithm is as good as 
other algorithms but no better on a bam. 
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EMPIRICAL results 


We have programmed the algorithm and tested it against the top-down 
and bottom-up parsers evaluated by Griffiths and Petrick [GP 65]. These 
are the back-tracking algorithms (section II!) whose time requirements 
can be exponential. However, they also can do well on some grammars, 
and they have both been used in numerous compi ler-compi lers, so it wi I I 
be interesting to compare our algorithm with them. 

The Griffiths and Petrick data is not in terms of actual running 
times, but in terms of "primitive operations." They have expressed 
their algorithms as sets of non-determi ni st i c rewriting rules for a 
Turing-machine- 1 ike device. Each application of one of these is a 
primitive operation. We have chosen as our primitive operation the act 
of adding a state to a state set (or attempting to add one which is 
already there). One can see from the data that this is comparable to 
what Griffiths and Petrick use. 

We compare the algorithms on 7 different grammars. We did not use 
2 of their examples because the exact grammar they used was not given. 
For the first four, Griffiths and Petrick were able to find closed 
form expressions for their results, so we did also (page 118). BU 
and TO are the bottom-up and top-down algorithms respectively, and 
SBU and STD are their selective versions. It is obvious from these 
results that SBU is by far the best of the other algorithms, and the 
rest of their data bears this out. Therefore we wi 1 1 compare our 

algorithm to SBU only. We used our algorithm with k = 0. The two are 



comparable on SI. S2, and G3, rhe simple grammars, but on G*. which is 

very ambiguous, ours is clearly super! or-n to n , 

For the next 3 grammars we present only the raw data (pages I 19-121). 

we have also included the data from [GP 65] on PA, the pred.ctive 

analyzer without path elimination [KO 63], On the propositional calculus 

grammar , PA seems to be running in time n 2 while both SBU and ours are 

in-time n, with ours a little faster. Grammar GBE produces two kinds 

Of. behavior. All three algorithms go up linearly with the number of 

■Vs, With SBU using a considerably higher constant coefficient. 

However, PA and SBU go up exponentially with the number of ed s, while 

ours goes up as the square. Grammar NSE I s qui te simp le, and each 

algorithm takes time n with the same coefficient. 

So we conclude that our algorithm is clearly superior to the 

. -+h mc l+ oerforms as well as the best of them on 
back-tracking algorithms. It perrorms as 

all seven grammars, and is substantially faster or ome. 

There are at least four distinct general context-free algorithms 

besides ours — TD, BU, Kasami's n 2 , and Cocke's n 3 . We have shown so 

far in this paper that our algorithm achieves time bounds which are as 

good as or better than those of any of these algorithms. However, we 

are also interested in how our algorithm compares with these algorithms 

in a practical sense, not just at an upper bound. 

• * „ i -oeiii+q in this section which 

We have just presented some empirical results 

indicate that our algorithm is better than TD and BU. Furthermore, 

algorithm must be superior to Cocke's since his always achieves 

upper bound of n 3 . This leaves Kasami 's. His algorithm [KT 



ac+ua I 
easi ly 
that i 
ours . 
parse 
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ly described as an algorithm for unambiguous grammars, but it 
be extended to a general algorithm. In this form we suspect 
twill have an n 3 bound in general and wi I I be n 2 as often as 
We are very curious about the class of grammars that it can 

in time n . 


can 



root : S Ab 

A -> a I Ab 

G3 

root: S -*■ ablaSb 


Grammar 

Sentence 

JD 

STD 

G 1 

u n 

ab 

(n 2 +7n+2)/2 

(n 2 +7nt2)/2 

G2 

n u 

a b 

3n+2 

2nt2 

G3 

n, n 
a b 

5n- 1 

5n-l 

G4 

ab n cd 

'\.2 n+6 

. ~n+2 

%2 


root : S - aB 

B aBl b 

G4 

root : S ** AB 

A -► a I Ab 
B be I bB I Bd 


BU 

SBU 

Ours 

9n+5 

9n+5 

4n+7 

|-2 n +7 

4n+4 

4n+4 

1 -2 n_l -5 

6n 

6n+4 

^2 n+5 

( n^+2 1 n 2 +46n+ 1 5 ) /3 

1 8n+8 
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Proposi-t-io^l Calculus Grammar 
root : F -*■ C I S I pI U 

C -*> U =3 U 
u -> (F) kuk 

L -*• L ' I p I q I r 
s+uvsluvu 
p + U A plu A U 


Sentence 

Lenqth 

pa 

SBU 

Ours 

p 

1 

14 

18 

28 

(p A q) 

5 

89 

56 

68 

(p'Aq) VrVpVq' 

13 

232 

185 

148 

p 3 ( (q 3 Mr 1 V (p A 
3 (q * V r)) 

26 

712 

277 

277 

-O (no ( p • A (q V r) A p')) 

17 

1955 

223 

141 

( (p A q) V (q A r) V (r A p' )) 
3 M( p' V q' ) A (r 1 V p) ) 

38 

2040 

562 

399 



root: X -*• alxbl Ya 

Y - cl YdY 


Sentence k enq — 

ededea ° 


4 

ededeab 

10 

JO 

ededeab 

16 

, ,200 
ededeab 

206 

4 

(ed) eabb 

12 

(ed) 7 eabb 

18 

(ed)^eabb 

20 


Grammar GRE 


PA 

SBU 

35 

52 

75 

92 

99 

152 

859 

2052 

617 

526 

24352 

16336 

86139 

54660 


» 


Ours 

33 

45 

63 

633 

79 

194 

251 



Grammar NSE 


root : S - AB 

A -*■ a I SC 
B ■> b 1 08 
C -*■ c 
D d 


Sentence 

Lenqth 

SBU 

Ours 

adbcddb 

7 

43 

44 

•5 34 

ad bcbcd bed b 

18 

1 ! 1 

108 

2 5 3 

adbed bed bed b 

19 

I 17 

1 14 

0 ) 

CL 

00 

cr 

20 

120 

123 

^3 2 4 

a(bc)"d (bed) dbed b 

24 

150 

141 

2 3 

a (bed) dbed beb 

16 

100 

95 
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XVI 


THE PRACTICAL USE OF THE ALGORISM 


So far in this paper, we have i ntroduced an algorithm (and a couple 
of variations of it,, and we have studied in detail its focal properties. 
Now we ask the questions, what is the practical value of the algorithm 
in what areas and in what form can it best be put to use? 

AS we mentioned in the introduction, the two main uses of context- 
free languages are in natural language processing and in programming 
language implementation. * .Ml restrict our attention to the latter 
area because we are not fami liar with the practical aspects of 
computational linguistics, and because it is compi ler work which 

originally motivated us to study parsing. 

First we ask, what impact will our algorithm have on the parsing 

done in production compilers for existing programming languages? The 
answer is, practically none. Production compilers require -easing 

time proportional to n with a fairly low coefficient of n. if the 

syntax of the language is fixed and the available programming effort is 
large, the parsing can be hand coded for the particular language. Th.s 
probably make it faster than or at least as fast as our algorithm 

or in fact any automatic parser. 

The value of any parsing algorithm is for use in compiler-writing 
languages [BM 62 ] or similar devices where the programmer i s experi- 
menting with the syntax of the language as he implements it, or wants 
to be able to implement many different languages easily, or for some 
other reason can't afford to code the parser by hand. Here he is 



wiMing to give up a little in speed to obtain the advantages of having 
to supply only the BNF of the language in order to get a parser. 

It is here that many of the time n algorithms which we mentioned 
in section III are used. Host of these, except for Knuth's algorithm, 
have been used (or have been tried) in compi ler systems of one form or 
another. They all share at least one important drawback. Most 
programming languages do not readily fall into the class of grammars 
which the algorithm will handle, and consequently much fiddling must 
be done with the grammar to get the parser to accept it. This seems 
not to be the case with the LR(k) algorithm. It seems to be the case 
that most programming languages naturally have LRd) grammars, or maybe 
LR<2) in a few places. And the one programming language which we have 
discovered that is not LR (k) (ignoring languages which are ambiguous) 

i S compi iable by our algorithm quite easi ly. 

So it seems that the compiled version of the LR(k) algorithm and 

the compi led version of ours which is more or less an extension of the 

- „ _ x -a i i ov i at i na this problem of having to 
LR(k), hold some promise of alleviaTi g k 

fiddle with the grammar. The coefficient of n is also reasonable 
these algorithms. It is very low for the LRd) algorithm) we estimate 
that it is about as good as a hand-coded parser. And given the pruning 
process, the coefficient for our compiled algorithm may go up only as 

much as the grammar deviates from being LRd). 

it is probably important to say here that we recommend using a 
look-ahead of I with both compi led algorithms. Implementing the full 
look-ahead for all k would cost more in programming time and efficiency 
than could be gained from the off chance grammar which might require a 
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,ar 9 e k. It tight be worthwhi le to implement the LROO algorithm to 
include a look-ahead of I or 2, but no more than this is needed. And 
a look-ahead of I at the most is needed for our algorithm, because it 
can compile a much larger class of grammars than the LR(k) algorithm, 
so the extra look-ahead is much less likely to be needed. 

This points out a possible advantage of our algorithm over just 
the LROO algorithm, even if we are working only with LR(k> grammars. 

If we have a grammar whi ch i s basical ly LR( I ) everywhere except in one 
or two "places" (Algol is this way), we would haye to do one of three 
things if we are using only the LR(k) algorithm. 

(1) Change the grammar so it is LR(I) everywhere. 

(2) Use an LR(2> parser. This, however, means losing efficiency 
everywhere because of the 2 character look-ahead. 

(3) Modify the LR(k> algorithm so that part of a grammar could 
do „e with a look-ahead of I and part with 2. This has not 

been done. 

However, if we are using our algorithm, we simply run it with a look ahead 
of I, and the few places where a look-ahead of two would have been needed 
by an LR(k> parser wi I I be taken care of automatically with only as 

X is necessary. Grammar LR2 (page 125) ' s 

much loss of efficiency as is necessary. 

i+ in far- i e cT 'he problem which makes some 

such an LR (2) grammar. It in fac, ec , v 

, ipm Think of Pd Ion 1 as an assignment statement 

Algol grammars LR(2). ininK ot pr 

. . numhpr of labels on such a 

and production 2 as allowing an arbi .ary number or 


statement . 



125 


Grammar LR2 


root: R - 1 I : = > 1 2 L :R 

L -*• 3 I 

Sentences: ( I = > (n 1 D 


S Q 0 -*• -R-t “* S 0 
R -► . I :=l s o 

R "*■ . L :R "! s q 

L -*• • I : S 0 

S j R I • : = ' "* ^0^3 

L * I • : S 0 ,S 3 

S 2 R -*• ' • :=> “* S o ,S 3 

R L. :R ^ ^0'^3 


Condition 


:, 3 = s q 

:,3=S 3 


R ■* t : • = ! *• S o ,:s 3 

R L:.R ^ V S 3 

R . I :=> -* s 3 

R -*• . L:R -t S 3 


Act i on Go To 

0, 1,2,3 

( 1 , 0 ) 

(3,0) S. 


(2,0) S 2 

(2.0) S 2 

(1.1) 

(2.1) s z 

1,2,3 

( 1 , 2 ) s , 

( 1 , 0 ) 

(3,0) S 


L - .1 
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R -> 1 : = . I "• S 0 ,S 3 

S c R I : =l S 0 ,S 3 

0 


Rendition 


Action 

(1,3) 


Go To 


S 


5 


^ 1=s o- 

H, l=S 3 


( 0 , 0 ) 

( 2 , 2 ) 


S, R L:R. 4 S 0 ,S 3 

b 


s ? 0 -*■ R--* A s o 

Sg ^ -*■ R^ ■* S Q 


^ 2 =S 0 

-l, 2 =S 3 


( 0 , 0 ) 

( 2 , 2 ) 


(0,1) S E 


HALT 
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Pruned Code 


S 


0 


s 


2 


S 


3 


S 


S 


S 


6 


Condition 


^ l=S 0 

h,i=s 3 

-»> 2=s 0 

H,2=S 3 


Action 

1,2 

( 1 , 0 ) 

( 2 , 0 ) 

( 1 , 1 ) 

( 2 , 1 ) 

1,2 

( 1 , 2 ) 

( 1 , 0 ) 

( 2 , 0 ) 

(1,3) 

HALT 

( 2 , 2 ) 

HALT 

( 2 , 2 ) 


Go To 




Grammar EXPCnl 


R -► A. 

i 

a i * c J A i 
A. * C.B.Id. 

B c ,B. I d. 
i J ' ' 


Example: EXP(2) 
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There is one large drawback to both the LR(k) algorithm and our 
algorithm in their compNed forms, however. That is that in generating 
all possible state sets at compile time, we may have to generate such 
a large number that we exceed the space of the machine on large 
grammars. This can be stated more precisely as follows: The space 

required for the recognition process is proportional to n, but in the 
compiled version of the algorithm, the space used depends on the size 
of the grammar as well as on the size of the string. And we have no 
guarantee that it depends linearly on the size of the grammar. In 
fact, page 128 contains a class of LR(O) grammars discovered by John 
Reynolds for which the number of state sets generated by Knuth's 
compilation algorithm goes up exponentially with the size of the 
grammar . 

Furthermore, [Ko 67] reports that in trying Knuth's compilation 
algorithm on a grammar for Algol, the number of state sets generated 
blew up. He presents a method for factoring an LR ( I ) grammar in 
pieces, applying the compilation algorithm to each of these, and then 
connecting the pieces into one parser. He claims that this cuts down 
on the number of state sets generated. We suspect that there are other 
small modifications in both Knuth's and our compilation algorithms 
which will guarantee a linear growth in the number of state sets with 
respect to the grammar for a large practical set of grammars. However, 
until this is accomplished, the compiled versions of these algorithms 
will not be of any practical value. 

Now, what about the original version? It is obviously useful in 
comp i lei — writing systems where the speed of the algorithm is not as 
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important os the fact that it 


accept any grammar without modifications 


l II! ian« . 

X +hflm Here it is a matter of choice. 
and won't run too slowly on any of them. 

un ti I the prohiems which we have mentioned are removed from Knuth's and 
our compi led algorithms, one has the choice between a large coeffcent 
of „ or a large a^unt of fiddling needed to get the grammar accepted 

by the recognizer. 

There is another use to which the original version 
su i ted. That is the extendible language systems [3P 67] which are being 
puilt now. These systems have the property that the syntax of the source 
language may be changed or extended as part of the program itself. Many 
Of them allow syntax extensions to be made at any point in the program, 
so as soon as some of them become conversational [St 68], then syntax changes 

Will be made capriciously and frequently. 

These systems will not tolerate a costly compilation 
parser each time a change is ^de in the syntax. The time n algorithms 
Which are in use no. al, require such compilations (in one form or 
another, and unless they can be altered to allow incremental 
recompilation when small changes are made, they may be useless in 
extendible systems. Thus if such systems 

extension, they M y haye to go to hon-compi led algorithms. In this 

be the best It will almost certainly do all the prac i 
class ours seems to be the besT. 

. - s net any n 2 or n^ grammars, 

grammars it gets i n ti me n and , f i , s get 

i+wil , do at least as well as any c ,r algorithm. The results ,n 
Section XV, also indicate that our constant coefficients are as good 
as or better than other algorithms. Krutar [Kr 68] is currently 
implementing such an extendible language system using our aigorithm. 



We should mention here one possible disadvantage of either version 
of our algorithm that is not shared by the time n algorithms. Our 
algorithm, or any of the others, may be caused to output a parse tree 
for the string it has recognized when it is finished. And in fact, 
some compiler-writing systems do work this way; the output of the 
parsing phase is a tree, which is then processed by the next phase of 
the compiler. Some other systems [BM 62], however, do not build up the 
whole parse tree before the next phase is called, but build up pieces 
of the tree and pass these on to the next phase, which works on them 
before the parsing can continue. Some systems even call the next phase 
every time a parsing decision is made [Fe 66]. 

All the time n algorithms (including Knuth's) are set up so that 
every time a parsing decision is made, it is indeed going to be part 
of the final parse tree for the sentence. This is not true of our 
algorithm. Because of the fact that we carry along all possible parses 
as we go, many parsing decisions (completer operations) are made which 
later turn out to be extraneous. For this reason, we can't afford to 
call the next phase every time a parsing decision is made. 

This problem can be remedied in the following way: Any time that 

a completer operation i s to be performed, it can be determined directly 
whether or not it is known then that this decision will be part of the 
final parse. If the final state causing the completer is the only one 
in its state set which matches the next k symbols, then it will be part 
of the final parse. If there are two or more such states, then the 
states that they produce by the completer can be marked and the parsing 
Later, as soon as a condition is reached where the 


can conti nue. 
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' . , c+a+es which were produced by one of 

current state set contains only states 

.. +ha+ final state is the one which represented 
those final states, then that final 

a n pv+ phase can be cal led for it. 
the correct parsing decision, and the next phase 

Even without this marking procedure, we can guarantee that oil 
parsing decisions wi II be oorrect at the time they are made for at 
least all LR(k) grammars for which we use a look-ahead of k. 

Our algorithm has the useful property that it can be modified to 
handle an extension of context-free grammars which contains the 
Kleene star notation. In this notation, 




means A may be rewritten as an arbitrary number 
followed by a "D" . It generates a language equ 

by 


(including 0) of "BC"'s 
valent to that generated 


A -* D BCA 


However, the parse structure given to the 
two grammars: 


language is different in the 



/! \ 


D 



.. - . . +he , eft cannot be obtained using context 

Structures like that on the left c 

• „ ;« n^pful The modification to 
tre e grammars at all, so this extension .susefu 

th . a I gori thm involves two additional operations: 

( | ) Any state of the form 


is rep I aced by 


A + a{.B)*Y f 

A -*■ a{B>*-Y f _ 


(2) Any state of the form 


A - a(B.}*Y f 


i s replaced by 


A -► a{ . 8)*Y f 
A -*• a{ 8)* • Y f 


Another possible advantage of oar algorithm over the t,me n 

algorithms (including is that we can handle amhigeons grammars, 

and |n tact handle .nvot them in time, — id ambiguity can . 

valuable in compi ,er-»riting languages to represent constructs -»« 

. . Lj.i. can be disambiguated easily Y 

are syntactically ambiguous, but wh.ch can be 

All of the time n algorithms wor 
using semantic considerations. All 

on subsets of the unambiguous grammars. 



mav no + want to implement the 

we should mention here that one may 

We snouiu thpre are certain!', 

u a soeci f > ed (Section VI). There 

algorithm exactly as we g . ve +ne quoted time and space bounds. 

other itnplemet lt a t i° ns that it i s possible to achi eve 

0ur i mp I ementat i on is merely to sho» 


these bounds. 



XVIII. 


further research and conclusion 


Some formal work still remains to be done on this algorithm: 

(1) Find a good algorithm-independent characterization of the time 
n grammars. Show that it includes bounded state and U LR(k) grammars. 

(2) Do the same for the space n grammars; do they include the 


unambiguous grammars? 

(3) Prove our conjecture (Section VII) that there is a language 
with inherent unbounded direct ambiguity, and that there is a language 
with inherent unbounded ambiguity which is 8DA. 


Much practical testing still is needed: 

(1) Implement and test the garbage collector (Section XIII). Are 

there any space problems with it in use? 

(2) Implement and test the compilation algorithm and the pruning 

procedure. In a practical sense, how large is the set of compilable 
grammars? What problems are there because the algorithm may loop? 

What is the coefficient of n like in practice. 

(3) Imbed either the compiled or the original algorithm in a 
compiler-writing or extendi b I e-language system. What are the interface 


prob I ems? 


Further research in the general field of context-free parsing 
might center on a number of different things. It may be possible to 
develop a general algorithm with an n 2 time bound. It seems unlikely 
that a general algorithm with a time n bound exists, although it has 
not been proven impossible. It would be worthwhi le to try to extend 
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.. h ran be done in time n, possibly to include 
the class of grammars which can 

alt unambiguous grammars. 

For +he researcher in automata theory or complexity theory, it 
w ou,d he interesting to try to obtain our n 2 or n resuits on a ham, or 
to try to prove that a general time n algorithm is impossible (for a 

bam at first try anyway). 

we discussed some compiler motivated research problems in 
Section xvl l . A modification for Knuth's (or our) compilation algorithm 
i£ needed which cuts down on the growth of the space with respect to 
the size of the gram™. For use in conversational extendible languages, 
an incremental recompi lation feature would be very valuable. Perhaps 
a completely different compilation algorithm should be dev.sed which 
either overcomes some of the problems of the others or oovers a wider 

class of grammars. 

I n conclusion let us emphasize that our algorithm not only matches 
or surpasses the best previous resuits for times n 3 (Younger), n 
.Kasami, and n (Knuth), but it does this with one single algorithm 
which does not have to have specified to it the Cass of graders ,t .s 

operating on. in other words, Knuth-s a.gorithm worts only on LR(t) 

, • _ hn+ ours works on i*hem 

grammars and Kasami 's only on unambiguous ones, 

an and seems to do about as well as it can automatically. 

Thus we have developed a context-free parsing algorithm which is 

guite efficient in the formal sense, and also shows promise of being 

very useful in several applications related to the implementation of 

programming languages. 
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